[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:11:55.230000 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:42907
I20260709 13:11:55.231643 32224 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:11:55.232666 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:55.249760 32230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:55.252259 32224 server_base.cc:1061] running on GCE node
W20260709 13:11:55.249750 32231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:55.250111 32233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:56.458395 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:56.458629 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:56.458901 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602716458875 us; error 0 us; skew 500 ppm
I20260709 13:11:56.459844 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:56.466411 32224 webserver.cc:533] Webserver started at http://127.31.120.62:41163/ using document root <none> and password file <none>
I20260709 13:11:56.467285 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:56.467514 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:56.468040 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:56.473824 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "95e7a9a3054a42cebacdd54e674e71ed"
format_stamp: "Formatted at 2026-07-09 13:11:56 on dist-test-slave-pm98"
I20260709 13:11:56.481776 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.001s	sys 0.008s
I20260709 13:11:56.487177 32239 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:56.488447 32224 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:11:56.488816 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "95e7a9a3054a42cebacdd54e674e71ed"
format_stamp: "Formatted at 2026-07-09 13:11:56 on dist-test-slave-pm98"
I20260709 13:11:56.489217 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:56.554456 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:56.556093 32224 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:11:56.556583 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:56.629357 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:42907
I20260709 13:11:56.629489 32290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:42907 every 8 connection(s)
I20260709 13:11:56.637171 32291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:56.659281 32291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: Bootstrap starting.
I20260709 13:11:56.665258 32291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:56.667079 32291 log.cc:826] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: Log is configured to *not* fsync() on all Append() calls
I20260709 13:11:56.671568 32291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: No bootstrap required, opened a new log
I20260709 13:11:56.691895 32291 raft_consensus.cc:359] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER }
I20260709 13:11:56.692437 32291 raft_consensus.cc:385] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:56.692765 32291 raft_consensus.cc:740] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e7a9a3054a42cebacdd54e674e71ed, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.693674 32291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER }
I20260709 13:11:56.694182 32291 raft_consensus.cc:399] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:56.694476 32291 raft_consensus.cc:493] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:56.694814 32291 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:56.699034 32291 raft_consensus.cc:515] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER }
I20260709 13:11:56.699831 32291 leader_election.cc:304] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95e7a9a3054a42cebacdd54e674e71ed; no voters: 
I20260709 13:11:56.701926 32291 leader_election.cc:290] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:56.702317 32294 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:56.704612 32294 raft_consensus.cc:697] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 1 LEADER]: Becoming Leader. State: Replica: 95e7a9a3054a42cebacdd54e674e71ed, State: Running, Role: LEADER
I20260709 13:11:56.705622 32294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER }
I20260709 13:11:56.706633 32291 sys_catalog.cc:565] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:56.716864 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95e7a9a3054a42cebacdd54e674e71ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER } }
I20260709 13:11:56.717775 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:56.719344 32295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95e7a9a3054a42cebacdd54e674e71ed. Latest consensus state: current_term: 1 leader_uuid: "95e7a9a3054a42cebacdd54e674e71ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e7a9a3054a42cebacdd54e674e71ed" member_type: VOTER } }
I20260709 13:11:56.720238 32295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:56.722398 32302 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:56.735725 32302 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:56.738000 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:56.751624 32302 catalog_manager.cc:1383] Generated new cluster ID: d4b798914c7444a3b7f9d183494ffdfa
I20260709 13:11:56.751955 32302 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:56.777551 32302 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:56.779192 32302 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:56.809094 32302 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: Generated new TSK 0
I20260709 13:11:56.810149 32302 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:56.869536 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:56.879760 32313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:56.881697 32314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:56.884501 32224 server_base.cc:1061] running on GCE node
W20260709 13:11:56.884979 32316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:56.885996 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:56.886279 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:56.886490 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602716886468 us; error 0 us; skew 500 ppm
I20260709 13:11:56.887322 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:56.891510 32224 webserver.cc:533] Webserver started at http://127.31.120.1:35199/ using document root <none> and password file <none>
I20260709 13:11:56.892272 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:56.892518 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:56.892869 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:56.894418 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
format_stamp: "Formatted at 2026-07-09 13:11:56 on dist-test-slave-pm98"
I20260709 13:11:56.901319 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:11:56.906466 32321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:56.907431 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:11:56.907810 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
format_stamp: "Formatted at 2026-07-09 13:11:56 on dist-test-slave-pm98"
I20260709 13:11:56.908193 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:56.930222 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:56.932014 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:56.934867 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:56.940515 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:56.940894 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:56.941219 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:56.941464 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.036695 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:45931
I20260709 13:11:57.036767 32383 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:45931 every 8 connection(s)
I20260709 13:11:57.044517 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:57.053463 32388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:57.055756 32389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:57.066320 32224 server_base.cc:1061] running on GCE node
W20260709 13:11:57.067031 32391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:57.068203 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:57.068475 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:57.068696 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602717068675 us; error 0 us; skew 500 ppm
I20260709 13:11:57.069501 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:57.072824 32224 webserver.cc:533] Webserver started at http://127.31.120.2:34117/ using document root <none> and password file <none>
I20260709 13:11:57.073446 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:57.073680 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:57.074019 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:57.074563 32384 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42907
I20260709 13:11:57.075026 32384 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:57.075507 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "64724d62f48f482d8392d18f0c28441e"
format_stamp: "Formatted at 2026-07-09 13:11:57 on dist-test-slave-pm98"
I20260709 13:11:57.076238 32384 heartbeater.cc:507] Master 127.31.120.62:42907 requested a full tablet report, sending...
I20260709 13:11:57.079805 32256 ts_manager.cc:194] Registered new tserver with Master: 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931)
I20260709 13:11:57.082149 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:57.082829 32256 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56942
I20260709 13:11:57.086895 32396 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.087817 32224 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 13:11:57.088104 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "64724d62f48f482d8392d18f0c28441e"
format_stamp: "Formatted at 2026-07-09 13:11:57 on dist-test-slave-pm98"
I20260709 13:11:57.088397 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:57.106060 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:57.107386 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:57.109797 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:57.112947 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:57.113183 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.113423 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:57.113590 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.209764 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:43535
I20260709 13:11:57.210105 32458 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:43535 every 8 connection(s)
I20260709 13:11:57.219365 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:57.227845 32463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:57.228822 32462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:57.233237 32465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:57.234064 32459 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42907
I20260709 13:11:57.234452 32459 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:57.235435 32459 heartbeater.cc:507] Master 127.31.120.62:42907 requested a full tablet report, sending...
I20260709 13:11:57.236307 32224 server_base.cc:1061] running on GCE node
I20260709 13:11:57.237396 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:57.237654 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:57.237890 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602717237869 us; error 0 us; skew 500 ppm
I20260709 13:11:57.237879 32256 ts_manager.cc:194] Registered new tserver with Master: 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:57.238833 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:57.240334 32256 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56946
I20260709 13:11:57.246018 32224 webserver.cc:533] Webserver started at http://127.31.120.3:35947/ using document root <none> and password file <none>
I20260709 13:11:57.246556 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:57.246783 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:57.247071 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:57.248205 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "282413e372c846f18cd8586ab1b610c0"
format_stamp: "Formatted at 2026-07-09 13:11:57 on dist-test-slave-pm98"
I20260709 13:11:57.253257 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:11:57.256989 32470 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.258129 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:11:57.258437 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "282413e372c846f18cd8586ab1b610c0"
format_stamp: "Formatted at 2026-07-09 13:11:57 on dist-test-slave-pm98"
I20260709 13:11:57.258734 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:57.294282 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:57.295740 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:57.298637 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:57.301951 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:57.302160 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.302498 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:57.302672 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:57.407749 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:38427
I20260709 13:11:57.407924 32532 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:38427 every 8 connection(s)
I20260709 13:11:57.424263 32533 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42907
I20260709 13:11:57.424636 32533 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:57.425573 32533 heartbeater.cc:507] Master 127.31.120.62:42907 requested a full tablet report, sending...
I20260709 13:11:57.427685 32256 ts_manager.cc:194] Registered new tserver with Master: 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:57.428517 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014932501s
I20260709 13:11:57.429208 32256 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56952
I20260709 13:11:57.483124 32256 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56964:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:57.486716 32256 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:57.583956 32424 tablet_service.cc:1511] Processing CreateTablet for tablet 03a50df29aed4877a0ba0933aa41df65 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:57.585696 32424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a50df29aed4877a0ba0933aa41df65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.594933 32347 tablet_service.cc:1511] Processing CreateTablet for tablet adb7cbb23e5a42a8a6453b1d31043b9f (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:57.595177 32349 tablet_service.cc:1511] Processing CreateTablet for tablet 03a50df29aed4877a0ba0933aa41df65 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:57.602262 32349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a50df29aed4877a0ba0933aa41df65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.603502 32423 tablet_service.cc:1511] Processing CreateTablet for tablet f95c9e5474f2449eb399be6862245506 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:57.604645 32346 tablet_service.cc:1511] Processing CreateTablet for tablet 03e558b1bf714079b1a62babadfb8468 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:57.609264 32347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb7cbb23e5a42a8a6453b1d31043b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.609323 32348 tablet_service.cc:1511] Processing CreateTablet for tablet f95c9e5474f2449eb399be6862245506 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:57.607959 32345 tablet_service.cc:1511] Processing CreateTablet for tablet 675e3147a0224eb5acf7c60137eab300 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:57.614542 32348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95c9e5474f2449eb399be6862245506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.615849 32422 tablet_service.cc:1511] Processing CreateTablet for tablet adb7cbb23e5a42a8a6453b1d31043b9f (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:57.618600 32423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95c9e5474f2449eb399be6862245506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.621054 32422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb7cbb23e5a42a8a6453b1d31043b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.618548 32421 tablet_service.cc:1511] Processing CreateTablet for tablet 03e558b1bf714079b1a62babadfb8468 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:57.624131 32346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e558b1bf714079b1a62babadfb8468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.626792 32421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e558b1bf714079b1a62babadfb8468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.628014 32345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675e3147a0224eb5acf7c60137eab300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.632285 32420 tablet_service.cc:1511] Processing CreateTablet for tablet 675e3147a0224eb5acf7c60137eab300 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:57.634269 32420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675e3147a0224eb5acf7c60137eab300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.669440 32344 tablet_service.cc:1511] Processing CreateTablet for tablet b49d887b268e41bea79aac32eac873ab (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:57.671092 32344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b49d887b268e41bea79aac32eac873ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.677734 32497 tablet_service.cc:1511] Processing CreateTablet for tablet f95c9e5474f2449eb399be6862245506 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:57.683164 32496 tablet_service.cc:1511] Processing CreateTablet for tablet adb7cbb23e5a42a8a6453b1d31043b9f (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:57.691769 32497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95c9e5474f2449eb399be6862245506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.684099 32498 tablet_service.cc:1511] Processing CreateTablet for tablet 03a50df29aed4877a0ba0933aa41df65 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:57.691635 32495 tablet_service.cc:1511] Processing CreateTablet for tablet 03e558b1bf714079b1a62babadfb8468 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:57.694310 32494 tablet_service.cc:1511] Processing CreateTablet for tablet 675e3147a0224eb5acf7c60137eab300 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:57.701886 32496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb7cbb23e5a42a8a6453b1d31043b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.703476 32419 tablet_service.cc:1511] Processing CreateTablet for tablet b49d887b268e41bea79aac32eac873ab (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:57.705755 32419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b49d887b268e41bea79aac32eac873ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.720808 32493 tablet_service.cc:1511] Processing CreateTablet for tablet b49d887b268e41bea79aac32eac873ab (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:57.735073 32495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e558b1bf714079b1a62babadfb8468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.737478 32494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675e3147a0224eb5acf7c60137eab300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.744380 32498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a50df29aed4877a0ba0933aa41df65. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:57.746619 32304 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:57.753361 32493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b49d887b268e41bea79aac32eac873ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.760939 32348 tablet_service.cc:1511] Processing CreateTablet for tablet 6b3b5f1d217a4782a3b6a3c322f1da77 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:57.762648 32348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b3b5f1d217a4782a3b6a3c322f1da77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.785662 32552 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:57.800887 32492 tablet_service.cc:1511] Processing CreateTablet for tablet 6b3b5f1d217a4782a3b6a3c322f1da77 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:57.802294 32492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b3b5f1d217a4782a3b6a3c322f1da77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.814254 32551 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:57.832964 32424 tablet_service.cc:1511] Processing CreateTablet for tablet 6b3b5f1d217a4782a3b6a3c322f1da77 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:57.836422 32345 tablet_service.cc:1511] Processing CreateTablet for tablet c30c4c3dc7874a8d9cd18ce2f3130c46 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:57.837527 32424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b3b5f1d217a4782a3b6a3c322f1da77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.838150 32345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c30c4c3dc7874a8d9cd18ce2f3130c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.840205 32423 tablet_service.cc:1511] Processing CreateTablet for tablet c30c4c3dc7874a8d9cd18ce2f3130c46 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:57.841432 32423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c30c4c3dc7874a8d9cd18ce2f3130c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.851117 32418 tablet_service.cc:1511] Processing CreateTablet for tablet 71075fda0d764cfeb3d9bb99f121c3da (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:57.854550 32418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71075fda0d764cfeb3d9bb99f121c3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.845553 32346 tablet_service.cc:1511] Processing CreateTablet for tablet 71075fda0d764cfeb3d9bb99f121c3da (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:57.861356 32346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71075fda0d764cfeb3d9bb99f121c3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.864713 32552 tablet_bootstrap.cc:654] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:57.879433 32553 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:57.855728 32493 tablet_service.cc:1511] Processing CreateTablet for tablet 71075fda0d764cfeb3d9bb99f121c3da (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:57.853840 32497 tablet_service.cc:1511] Processing CreateTablet for tablet c30c4c3dc7874a8d9cd18ce2f3130c46 (DEFAULT_TABLE table=test-workload [id=c9b366b186fd4346be6093332e2c85b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:57.883715 32493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71075fda0d764cfeb3d9bb99f121c3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.885970 32497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c30c4c3dc7874a8d9cd18ce2f3130c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:57.908838 32553 tablet_bootstrap.cc:654] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:57.921351 32551 tablet_bootstrap.cc:654] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:57.939013 32552 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:57.939695 32552 ts_tablet_manager.cc:1403] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.160s	user 0.018s	sys 0.020s
I20260709 13:11:57.942673 32552 raft_consensus.cc:359] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.943487 32552 raft_consensus.cc:385] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:57.943815 32552 raft_consensus.cc:740] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.944633 32552 consensus_queue.cc:260] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.945482 32553 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:57.946024 32553 ts_tablet_manager.cc:1403] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.010s
I20260709 13:11:57.950142 32553 raft_consensus.cc:359] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.954352 32553 raft_consensus.cc:385] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:57.954854 32553 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.955439 32551 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:57.955996 32551 ts_tablet_manager.cc:1403] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.166s	user 0.018s	sys 0.018s
I20260709 13:11:57.956662 32553 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.959643 32459 heartbeater.cc:499] Master 127.31.120.62:42907 was elected leader, sending a full tablet report...
I20260709 13:11:57.960369 32552 ts_tablet_manager.cc:1434] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.010s
I20260709 13:11:57.961665 32552 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
W20260709 13:11:57.969632 32460 tablet.cc:2406] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:11:57.972479 32552 tablet_bootstrap.cc:654] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:57.958736 32551 raft_consensus.cc:359] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.979832 32551 raft_consensus.cc:385] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:57.980192 32551 raft_consensus.cc:740] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.980947 32551 consensus_queue.cc:260] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.984483 32533 heartbeater.cc:499] Master 127.31.120.62:42907 was elected leader, sending a full tablet report...
I20260709 13:11:57.985495 32553 ts_tablet_manager.cc:1434] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.016s
I20260709 13:11:57.986107 32552 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:57.986495 32553 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:57.986620 32552 ts_tablet_manager.cc:1403] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.009s
I20260709 13:11:57.989678 32552 raft_consensus.cc:359] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.990566 32552 raft_consensus.cc:385] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:57.990939 32552 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.991705 32551 ts_tablet_manager.cc:1434] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.004s
I20260709 13:11:57.991822 32552 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:57.992434 32384 heartbeater.cc:499] Master 127.31.120.62:42907 was elected leader, sending a full tablet report...
I20260709 13:11:57.992681 32551 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:57.996604 32553 tablet_bootstrap.cc:654] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:57.996995 32552 ts_tablet_manager.cc:1434] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:11:57.998128 32552 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.018831 32552 tablet_bootstrap.cc:654] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.029070 32553 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.029847 32553 ts_tablet_manager.cc:1403] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.021s
I20260709 13:11:58.032516 32552 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.032999 32552 ts_tablet_manager.cc:1403] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.006s
I20260709 13:11:58.033044 32553 raft_consensus.cc:359] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.033957 32553 raft_consensus.cc:385] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.034258 32553 raft_consensus.cc:740] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.035378 32551 tablet_bootstrap.cc:654] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.035133 32553 consensus_queue.cc:260] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.036118 32552 raft_consensus.cc:359] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.036842 32552 raft_consensus.cc:385] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.037158 32552 raft_consensus.cc:740] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.037550 32553 ts_tablet_manager.cc:1434] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:58.037868 32552 consensus_queue.cc:260] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.038400 32553 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.046561 32553 tablet_bootstrap.cc:654] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.047652 32552 ts_tablet_manager.cc:1434] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:11:58.051008 32552 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.058384 32552 tablet_bootstrap.cc:654] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.059679 32551 tablet_bootstrap.cc:492] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.060184 32551 ts_tablet_manager.cc:1403] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.000s
I20260709 13:11:58.063016 32551 raft_consensus.cc:359] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.063732 32551 raft_consensus.cc:385] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.064028 32551 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.064738 32551 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.069408 32553 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.069895 32553 ts_tablet_manager.cc:1403] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.007s
I20260709 13:11:58.072736 32553 raft_consensus.cc:359] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.073422 32553 raft_consensus.cc:385] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.073709 32553 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.074424 32553 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.078024 32553 ts_tablet_manager.cc:1434] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:11:58.078975 32553 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.085789 32553 tablet_bootstrap.cc:654] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.087190 32551 ts_tablet_manager.cc:1434] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:11:58.088270 32551 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.091012 32552 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.091976 32552 ts_tablet_manager.cc:1403] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.015s
I20260709 13:11:58.092092 32553 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.096455 32553 ts_tablet_manager.cc:1403] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.003s
I20260709 13:11:58.096028 32552 raft_consensus.cc:359] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.097359 32552 raft_consensus.cc:385] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.097689 32552 raft_consensus.cc:740] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.097851 32551 tablet_bootstrap.cc:654] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.098500 32552 consensus_queue.cc:260] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.099334 32553 raft_consensus.cc:359] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.100485 32553 raft_consensus.cc:385] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.100772 32553 raft_consensus.cc:740] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.101799 32552 ts_tablet_manager.cc:1434] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:11:58.101471 32553 consensus_queue.cc:260] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.102087 32560 raft_consensus.cc:493] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.102694 32552 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.102557 32560 raft_consensus.cc:515] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.103832 32553 ts_tablet_manager.cc:1434] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:58.104734 32553 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.111311 32552 tablet_bootstrap.cc:654] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.116014 32553 tablet_bootstrap.cc:654] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.122610 32552 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.123162 32552 ts_tablet_manager.cc:1403] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:11:58.124507 32560 leader_election.cc:290] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.132457 32552 raft_consensus.cc:359] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.133483 32552 raft_consensus.cc:385] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.133796 32552 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.134629 32552 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.136770 32553 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.144191 32553 ts_tablet_manager.cc:1403] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.014s
I20260709 13:11:58.146885 32553 raft_consensus.cc:359] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.147579 32553 raft_consensus.cc:385] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.148051 32553 raft_consensus.cc:740] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.148809 32553 consensus_queue.cc:260] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.139528 32551 tablet_bootstrap.cc:492] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.150097 32551 ts_tablet_manager.cc:1403] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.004s
I20260709 13:11:58.153007 32551 raft_consensus.cc:359] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.156705 32551 raft_consensus.cc:385] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.157104 32551 raft_consensus.cc:740] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.156350 32553 ts_tablet_manager.cc:1434] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.005s
I20260709 13:11:58.162506 32553 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.158964 32552 ts_tablet_manager.cc:1434] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:11:58.171319 32553 tablet_bootstrap.cc:654] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.171787 32551 consensus_queue.cc:260] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.176990 32552 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.186307 32551 ts_tablet_manager.cc:1434] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:11:58.190233 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.191138 32508 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.191088 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.193636 32552 tablet_bootstrap.cc:654] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:58.194445 32324 leader_election.cc:343] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.195331 32324 leader_election.cc:304] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:58.196239 32560 raft_consensus.cc:2804] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.196638 32560 raft_consensus.cc:493] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.192116 32551 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.197041 32560 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.200618 32553 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.201113 32553 ts_tablet_manager.cc:1403] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.011s
I20260709 13:11:58.203926 32551 tablet_bootstrap.cc:654] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.204376 32553 raft_consensus.cc:359] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.205217 32553 raft_consensus.cc:385] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.204494 32560 raft_consensus.cc:515] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.205559 32553 raft_consensus.cc:740] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.206723 32553 consensus_queue.cc:260] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.213569 32557 raft_consensus.cc:493] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.214128 32557 raft_consensus.cc:515] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.212076 32552 tablet_bootstrap.cc:492] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.219888 32560 leader_election.cc:290] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.220744 32560 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.218927 32557 leader_election.cc:290] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.222075 32560 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.225013 32560 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.221493 32552 ts_tablet_manager.cc:1403] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.005s
I20260709 13:11:58.229408 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.229622 32552 raft_consensus.cc:359] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.230202 32508 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.230450 32552 raft_consensus.cc:385] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.230836 32552 raft_consensus.cc:740] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.233939 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.237186 32558 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.226869 32553 ts_tablet_manager.cc:1434] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.005s
I20260709 13:11:58.237846 32558 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.245340 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.246472 32507 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.250499 32558 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:58.252506 32558 raft_consensus.cc:493] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.257704 32553 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.256544 32558 raft_consensus.cc:515] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.252600 32324 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:58.267529 32558 leader_election.cc:290] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:58.267984 32560 raft_consensus.cc:2804] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.268356 32560 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.268677 32560 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.270038 32508 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.258791 32552 consensus_queue.cc:260] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.275230 32434 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.281755 32433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.282586 32433 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.286111 32551 tablet_bootstrap.cc:492] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.288831 32551 ts_tablet_manager.cc:1403] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.099s	user 0.021s	sys 0.036s
I20260709 13:11:58.296985 32560 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.300334 32551 raft_consensus.cc:359] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.301120 32551 raft_consensus.cc:385] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.301173 32560 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.302488 32433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.303913 32433 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.301584 32551 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.302415 32552 ts_tablet_manager.cc:1434] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.003s
I20260709 13:11:58.310204 32434 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.310250 32324 leader_election.cc:304] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:58.311395 32551 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.312520 32560 raft_consensus.cc:2804] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.314903 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.316262 32552 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.316560 32508 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.314015 32560 raft_consensus.cc:697] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:58.320978 32560 consensus_queue.cc:237] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.319868 32553 tablet_bootstrap.cc:654] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.330950 32433 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.333806 32324 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.338233 32560 raft_consensus.cc:2804] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.341778 32560 raft_consensus.cc:697] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:58.346175 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.346159 32560 consensus_queue.cc:237] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
W20260709 13:11:58.348865 32399 leader_election.cc:343] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.348654 32508 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.347952 32551 ts_tablet_manager.cc:1434] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.058s	user 0.011s	sys 0.012s
I20260709 13:11:58.351291 32433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.353722 32433 raft_consensus.cc:2393] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 282413e372c846f18cd8586ab1b610c0 in current term 1: Already voted for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in this term.
I20260709 13:11:58.352762 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.354449 32551 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.355875 32432 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 0.
I20260709 13:11:58.368711 32473 leader_election.cc:304] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.373432 32558 raft_consensus.cc:2804] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.373973 32558 raft_consensus.cc:493] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.374470 32558 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.380451 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.383775 32551 tablet_bootstrap.cc:654] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:58.386109 32400 leader_election.cc:343] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.386847 32400 leader_election.cc:304] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64724d62f48f482d8392d18f0c28441e; no voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:58.387246 32552 tablet_bootstrap.cc:654] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.388020 32557 raft_consensus.cc:2749] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:58.393319 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.393874 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.394650 32358 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 0.
I20260709 13:11:58.396133 32474 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0; no voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:58.409269 32553 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.409961 32553 ts_tablet_manager.cc:1403] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.156s	user 0.016s	sys 0.032s
I20260709 13:11:58.409916 32558 raft_consensus.cc:515] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.411553 32589 raft_consensus.cc:2749] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:58.412432 32558 leader_election.cc:290] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:58.412820 32552 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.413256 32552 ts_tablet_manager.cc:1403] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.098s	user 0.019s	sys 0.010s
I20260709 13:11:58.413705 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.414294 32432 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.413848 32553 raft_consensus.cc:359] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.414942 32553 raft_consensus.cc:385] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.415290 32553 raft_consensus.cc:740] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.415866 32552 raft_consensus.cc:359] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.414294 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:58.417117 32552 raft_consensus.cc:385] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.417455 32552 raft_consensus.cc:740] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.417510 32358 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.416074 32553 consensus_queue.cc:260] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.418548 32552 consensus_queue.cc:260] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.422966 32552 ts_tablet_manager.cc:1434] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:11:58.423182 32432 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 1.
I20260709 13:11:58.424137 32553 ts_tablet_manager.cc:1434] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:11:58.424643 32473 leader_election.cc:304] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.425055 32553 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.425765 32558 raft_consensus.cc:2804] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.426046 32552 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.427096 32358 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 1.
I20260709 13:11:58.427742 32558 raft_consensus.cc:697] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:11:58.428099 32551 tablet_bootstrap.cc:492] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.428551 32551 ts_tablet_manager.cc:1403] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.076s	user 0.022s	sys 0.004s
I20260709 13:11:58.428633 32558 consensus_queue.cc:237] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.432094 32551 raft_consensus.cc:359] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.432611 32552 tablet_bootstrap.cc:654] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.432968 32551 raft_consensus.cc:385] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.433348 32551 raft_consensus.cc:740] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.435575 32551 consensus_queue.cc:260] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.438388 32551 ts_tablet_manager.cc:1434] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:11:58.439515 32551 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.441376 32552 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.441732 32557 raft_consensus.cc:493] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.441916 32552 ts_tablet_manager.cc:1403] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.006s
I20260709 13:11:58.442214 32557 raft_consensus.cc:515] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.444602 32553 tablet_bootstrap.cc:654] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.445166 32552 raft_consensus.cc:359] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.445986 32552 raft_consensus.cc:385] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.446355 32552 raft_consensus.cc:740] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.446192 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b49d887b268e41bea79aac32eac873ab" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.447050 32358 raft_consensus.cc:2468] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 0.
I20260709 13:11:58.448468 32400 leader_election.cc:304] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.449277 32557 leader_election.cc:290] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.450019 32557 raft_consensus.cc:2804] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.450552 32557 raft_consensus.cc:493] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.451071 32557 raft_consensus.cc:3060] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.455255 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b49d887b268e41bea79aac32eac873ab" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.456385 32508 raft_consensus.cc:2468] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 0.
I20260709 13:11:58.463676 32551 tablet_bootstrap.cc:654] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.466102 32557 raft_consensus.cc:515] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.470098 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b49d887b268e41bea79aac32eac873ab" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.470772 32508 raft_consensus.cc:3060] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.472935 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b49d887b268e41bea79aac32eac873ab" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:58.473133 32552 consensus_queue.cc:260] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.473626 32358 raft_consensus.cc:3060] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.470729 32256 catalog_manager.cc:5697] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:58.475284 32552 ts_tablet_manager.cc:1434] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.002s
I20260709 13:11:58.476202 32552 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: Bootstrap starting.
I20260709 13:11:58.482707 32553 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.483115 32552 tablet_bootstrap.cc:654] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.483258 32553 ts_tablet_manager.cc:1403] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.017s
I20260709 13:11:58.484391 32358 raft_consensus.cc:2468] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 1.
I20260709 13:11:58.486137 32560 raft_consensus.cc:493] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.488265 32560 raft_consensus.cc:515] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.488924 32400 leader_election.cc:304] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.489559 32557 leader_election.cc:290] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.490178 32557 raft_consensus.cc:2804] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.487670 32508 raft_consensus.cc:2468] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 1.
I20260709 13:11:58.496102 32557 raft_consensus.cc:697] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:58.497004 32557 consensus_queue.cc:237] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.486027 32553 raft_consensus.cc:359] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.490373 32560 leader_election.cc:290] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.492355 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.498543 32434 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.498680 32553 raft_consensus.cc:385] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.493006 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.499022 32553 raft_consensus.cc:740] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.499503 32507 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.499768 32553 consensus_queue.cc:260] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.501328 32324 leader_election.cc:304] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:58.502074 32560 raft_consensus.cc:2804] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.502421 32560 raft_consensus.cc:493] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.502791 32560 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.504606 32552 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: No bootstrap required, opened a new log
I20260709 13:11:58.505072 32552 ts_tablet_manager.cc:1403] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.008s
I20260709 13:11:58.507709 32552 raft_consensus.cc:359] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.508566 32552 raft_consensus.cc:385] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.508821 32552 raft_consensus.cc:740] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.509704 32552 consensus_queue.cc:260] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.508852 32553 ts_tablet_manager.cc:1434] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:11:58.512138 32552 ts_tablet_manager.cc:1434] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:58.512804 32560 raft_consensus.cc:515] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.514504 32560 leader_election.cc:290] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.515487 32553 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: Bootstrap starting.
I20260709 13:11:58.516604 32551 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.516500 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.517072 32551 ts_tablet_manager.cc:1403] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.078s	user 0.022s	sys 0.010s
I20260709 13:11:58.517304 32434 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.520397 32551 raft_consensus.cc:359] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.521116 32551 raft_consensus.cc:385] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.521391 32551 raft_consensus.cc:740] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.522096 32551 consensus_queue.cc:260] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.522400 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.522989 32507 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.524654 32551 ts_tablet_manager.cc:1434] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:11:58.525154 32553 tablet_bootstrap.cc:654] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.525679 32551 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.534731 32507 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.536331 32324 leader_election.cc:304] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:58.537221 32560 raft_consensus.cc:2804] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.537709 32560 raft_consensus.cc:697] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:58.539240 32434 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.538950 32560 consensus_queue.cc:237] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.542621 32254 catalog_manager.cc:5697] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.545085 32551 tablet_bootstrap.cc:654] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.557461 32558 raft_consensus.cc:493] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.558116 32558 raft_consensus.cc:515] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.560338 32551 tablet_bootstrap.cc:492] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.560904 32551 ts_tablet_manager.cc:1403] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.035s	user 0.005s	sys 0.008s
I20260709 13:11:58.563920 32557 raft_consensus.cc:493] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.563620 32551 raft_consensus.cc:359] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.564412 32551 raft_consensus.cc:385] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.564378 32557 raft_consensus.cc:515] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.564819 32551 raft_consensus.cc:740] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.565600 32551 consensus_queue.cc:260] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.566103 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.566525 32557 leader_election.cc:290] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.567375 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
I20260709 13:11:58.567581 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.568217 32507 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 0.
I20260709 13:11:58.568893 32558 leader_election.cc:290] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:58.569796 32399 leader_election.cc:304] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.569892 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.568182 32359 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 0.
I20260709 13:11:58.570588 32434 raft_consensus.cc:2468] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 0.
I20260709 13:11:58.570611 32557 raft_consensus.cc:2804] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.591439 32557 raft_consensus.cc:493] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.591837 32557 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.594705 32254 catalog_manager.cc:5697] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:58.569908 32474 leader_election.cc:343] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.596393 32557 raft_consensus.cc:515] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.596222 32254 catalog_manager.cc:5697] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.600500 32473 leader_election.cc:304] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:58.601094 32557 leader_election.cc:290] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.588519 32551 ts_tablet_manager.cc:1434] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:11:58.600607 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.602015 32507 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.602661 32551 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.602864 32558 raft_consensus.cc:2804] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.603363 32558 raft_consensus.cc:493] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.603742 32558 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.607772 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:58.608405 32359 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.589948 32553 tablet_bootstrap.cc:492] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: No bootstrap required, opened a new log
I20260709 13:11:58.609841 32553 ts_tablet_manager.cc:1403] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: Time spent bootstrapping tablet: real 0.095s	user 0.009s	sys 0.003s
I20260709 13:11:58.610359 32551 tablet_bootstrap.cc:654] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.612721 32553 raft_consensus.cc:359] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.613554 32553 raft_consensus.cc:385] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.613873 32553 raft_consensus.cc:740] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.611269 32507 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 1.
I20260709 13:11:58.614653 32553 consensus_queue.cc:260] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.615388 32399 leader_election.cc:304] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.616258 32557 raft_consensus.cc:2804] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.616945 32557 raft_consensus.cc:697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:58.617738 32557 consensus_queue.cc:237] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.621016 32553 ts_tablet_manager.cc:1434] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:11:58.621784 32558 raft_consensus.cc:515] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.624305 32359 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 1.
I20260709 13:11:58.624553 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
W20260709 13:11:58.626385 32474 leader_election.cc:343] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.630190 32558 leader_election.cc:290] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:58.631309 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.631932 32434 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.640709 32434 raft_consensus.cc:2468] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 1.
I20260709 13:11:58.640439 32255 catalog_manager.cc:5697] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 0 to 1, leader changed from <none> to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.641839 32473 leader_election.cc:304] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:58.642675 32558 raft_consensus.cc:2804] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.643082 32557 raft_consensus.cc:493] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.643146 32558 raft_consensus.cc:697] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:11:58.643651 32557 raft_consensus.cc:515] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.644100 32558 consensus_queue.cc:237] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.646473 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" is_pre_election: true
W20260709 13:11:58.647742 32400 leader_election.cc:343] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.653837 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.654511 32507 raft_consensus.cc:2468] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 0.
I20260709 13:11:58.655730 32399 leader_election.cc:304] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:58.656764 32593 raft_consensus.cc:2804] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.657130 32593 raft_consensus.cc:493] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.657469 32593 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.661677 32551 tablet_bootstrap.cc:492] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.662278 32551 ts_tablet_manager.cc:1403] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.015s
I20260709 13:11:58.665046 32593 raft_consensus.cc:515] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.666994 32551 raft_consensus.cc:359] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.667817 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:58.668156 32551 raft_consensus.cc:385] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.668562 32551 raft_consensus.cc:740] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
W20260709 13:11:58.669057 32400 leader_election.cc:343] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:58.669744 32557 leader_election.cc:290] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.669355 32551 consensus_queue.cc:260] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.673426 32551 ts_tablet_manager.cc:1434] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:11:58.675230 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.676282 32507 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.676824 32551 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: Bootstrap starting.
I20260709 13:11:58.675698 32593 leader_election.cc:290] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.674377 32254 catalog_manager.cc:5697] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:58.672993 32253 catalog_manager.cc:5697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 0 to 1, leader changed from <none> to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.681893 32551 tablet_bootstrap.cc:654] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:58.682935 32507 raft_consensus.cc:2468] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 1.
I20260709 13:11:58.684141 32399 leader_election.cc:304] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:58.684947 32557 raft_consensus.cc:2804] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.689186 32557 raft_consensus.cc:697] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:58.690093 32557 consensus_queue.cc:237] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.697784 32551 tablet_bootstrap.cc:492] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: No bootstrap required, opened a new log
I20260709 13:11:58.698249 32551 ts_tablet_manager.cc:1403] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:11:58.700934 32551 raft_consensus.cc:359] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.701629 32551 raft_consensus.cc:385] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:58.701931 32551 raft_consensus.cc:740] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.702607 32551 consensus_queue.cc:260] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.704591 32551 ts_tablet_manager.cc:1434] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:58.711814 32255 catalog_manager.cc:5697] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 0 to 1, leader changed from <none> to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.747494 32560 raft_consensus.cc:493] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.748049 32560 raft_consensus.cc:515] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
W20260709 13:11:58.748893 32304 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:58.750566 32560 leader_election.cc:290] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.751345 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.752188 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.753017 32508 raft_consensus.cc:2393] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31ebbeb35c22427db66f1ca4fa3730c5 in current term 1: Already voted for candidate 64724d62f48f482d8392d18f0c28441e in this term.
I20260709 13:11:58.753950 32324 leader_election.cc:304] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:58.753814 32560 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:58.754812 32583 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.760970 32560 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:58.762071 32583 raft_consensus.cc:2749] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:58.781543 32576 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:58.802505 32613 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:58.825762 32613 raft_consensus.cc:493] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.826359 32613 raft_consensus.cc:515] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.828204 32613 leader_election.cc:290] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.828900 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.829190 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.829540 32432 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.829807 32508 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 0.
I20260709 13:11:58.829865 32613 raft_consensus.cc:493] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.830559 32324 leader_election.cc:304] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:58.831314 32583 raft_consensus.cc:2804] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:58.831063 32613 raft_consensus.cc:515] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.831717 32583 raft_consensus.cc:493] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.832094 32583 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.833106 32613 leader_election.cc:290] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.833858 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e" is_pre_election: true
I20260709 13:11:58.834649 32432 raft_consensus.cc:2393] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31ebbeb35c22427db66f1ca4fa3730c5 in current term 1: Already voted for candidate 282413e372c846f18cd8586ab1b610c0 in this term.
I20260709 13:11:58.834772 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0" is_pre_election: true
I20260709 13:11:58.836621 32324 leader_election.cc:304] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:58.837266 32613 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.838690 32583 raft_consensus.cc:515] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.842022 32583 leader_election.cc:290] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:58.843220 32613 raft_consensus.cc:2749] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:58.843217 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:58.844115 32508 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.844485 32558 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:58.857040 32558 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:58.847327 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:58.863965 32432 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:58.872773 32508 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.873898 32324 leader_election.cc:304] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:58.875792 32613 raft_consensus.cc:2804] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:58.876062 32432 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 1.
I20260709 13:11:58.876296 32613 raft_consensus.cc:697] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:58.877178 32613 consensus_queue.cc:237] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:58.887931 32253 catalog_manager.cc:5697] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.974162 32557 consensus_queue.cc:1048] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:58.995201 32557 consensus_queue.cc:1048] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:58.997277 32508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
mode: GRACEFUL
new_leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:11:58.998072 32508 raft_consensus.cc:606] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Received request to transfer leadership to TS 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:11:59.021492 32359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:50188
I20260709 13:11:59.022047 32359 raft_consensus.cc:606] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:59.030864 32508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:11:59.031482 32508 raft_consensus.cc:606] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:11:59.047544 32621 consensus_queue.cc:1048] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:59.050016 32432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:59626
I20260709 13:11:59.050598 32432 raft_consensus.cc:606] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Received request to transfer leadership to TS 282413e372c846f18cd8586ab1b610c0
I20260709 13:11:59.064333 32621 consensus_queue.cc:1048] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.074208 32558 raft_consensus.cc:993] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:11:59.074718 32621 raft_consensus.cc:1081] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:11:59.076892 32432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59616
I20260709 13:11:59.077696 32432 raft_consensus.cc:493] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.078169 32432 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.083581 32432 raft_consensus.cc:515] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.086439 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:59.086989 32359 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.087916 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b3b5f1d217a4782a3b6a3c322f1da77" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:59.088555 32508 raft_consensus.cc:3055] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:59.088940 32508 raft_consensus.cc:740] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:11:59.089870 32508 consensus_queue.cc:260] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.091293 32508 raft_consensus.cc:3060] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.093685 32359 raft_consensus.cc:2468] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:11:59.094483 32432 leader_election.cc:290] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:59.095018 32400 leader_election.cc:304] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:59.095983 32557 raft_consensus.cc:2804] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:59.096491 32557 raft_consensus.cc:697] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:59.097363 32557 consensus_queue.cc:237] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.098196 32508 raft_consensus.cc:2468] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:11:59.102658 32613 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.105623 32253 catalog_manager.cc:5697] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 1 to 2, leader changed from 282413e372c846f18cd8586ab1b610c0 (127.31.120.3) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 2 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:59.121873 32583 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.122434 32557 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.132961 32557 consensus_queue.cc:1048] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:59.149952 32608 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:59.157342 32583 raft_consensus.cc:993] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:11:59.157855 32613 raft_consensus.cc:1081] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:11:59.167049 32432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59604
I20260709 13:11:59.167768 32432 raft_consensus.cc:493] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.168213 32432 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.174171 32557 consensus_queue.cc:1048] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.177570 32432 raft_consensus.cc:515] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.183727 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:59.184315 32507 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.193699 32507 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:11:59.194855 32399 leader_election.cc:304] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:59.195745 32608 raft_consensus.cc:2804] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:59.196408 32608 raft_consensus.cc:697] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:59.196626 32432 leader_election.cc:290] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:59.197378 32608 consensus_queue.cc:237] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.197949 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:59.198882 32358 raft_consensus.cc:3055] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:59.199923 32358 raft_consensus.cc:740] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:59.201349 32358 consensus_queue.cc:260] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.204273 32358 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.206149 32593 raft_consensus.cc:993] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:11:59.206604 32593 raft_consensus.cc:1081] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Signalling peer 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:11:59.208357 32507 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:48008
I20260709 13:11:59.208892 32507 raft_consensus.cc:493] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.209242 32507 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.210479 32253 catalog_manager.cc:5697] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 1 to 2, leader changed from 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 2 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:59.216024 32507 raft_consensus.cc:515] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.211239 32358 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:11:59.218888 32507 leader_election.cc:290] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:11:59.219637 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:11:59.220203 32358 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.220870 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71075fda0d764cfeb3d9bb99f121c3da" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:59.221453 32432 raft_consensus.cc:3055] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:59.221781 32432 raft_consensus.cc:740] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:11:59.222534 32432 consensus_queue.cc:260] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.224056 32432 raft_consensus.cc:3060] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.226756 32358 raft_consensus.cc:2468] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:11:59.227888 32474 leader_election.cc:304] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:11:59.228677 32621 raft_consensus.cc:2804] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:59.229139 32621 raft_consensus.cc:697] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:11:59.229938 32621 consensus_queue.cc:237] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.231225 32432 raft_consensus.cc:2468] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:11:59.237301 32256 catalog_manager.cc:5697] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 1 to 2, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:59.281567 32558 raft_consensus.cc:993] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: : Instructing follower 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:11:59.282027 32621 raft_consensus.cc:1081] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Signalling peer 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:11:59.283720 32358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:50178
I20260709 13:11:59.284252 32358 raft_consensus.cc:493] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.284611 32358 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.291215 32358 raft_consensus.cc:515] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.293249 32358 leader_election.cc:290] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 2 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:11:59.294222 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:11:59.294557 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:11:59.294996 32432 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.295121 32507 raft_consensus.cc:3055] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:59.295480 32507 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:11:59.296294 32507 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.297484 32507 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.300276 32432 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 2.
I20260709 13:11:59.301450 32324 leader_election.cc:304] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:11:59.302160 32583 raft_consensus.cc:2804] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:59.302698 32583 raft_consensus.cc:697] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:11:59.303498 32583 consensus_queue.cc:237] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:11:59.304450 32507 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 2.
I20260709 13:11:59.310782 32256 catalog_manager.cc:5697] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 1 to 2, leader changed from 282413e372c846f18cd8586ab1b610c0 (127.31.120.3) to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:59.415113 32583 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.434737 32583 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:59.611586 32358 raft_consensus.cc:1275] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.613090 32593 consensus_queue.cc:1048] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.621280 32507 raft_consensus.cc:1275] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.622563 32557 consensus_queue.cc:1048] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.689693 32507 raft_consensus.cc:1275] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.690946 32608 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.698302 32358 raft_consensus.cc:1275] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.699784 32557 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.718752 32358 raft_consensus.cc:1275] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.720160 32621 consensus_queue.cc:1048] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.729243 32432 raft_consensus.cc:1275] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.731442 32621 consensus_queue.cc:1048] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.829788 32432 raft_consensus.cc:1275] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.831135 32583 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.833817 32507 raft_consensus.cc:1275] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:59.835866 32560 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:02.083279 32224 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:12:02.084146 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:02.105304 32432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:02.105912 32432 raft_consensus.cc:606] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Received request to transfer leadership to TS 282413e372c846f18cd8586ab1b610c0
I20260709 13:12:02.116657 32358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
mode: GRACEFUL
new_leader_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:50196
I20260709 13:12:02.117188 32358 raft_consensus.cc:606] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Received request to transfer leadership to TS 282413e372c846f18cd8586ab1b610c0
I20260709 13:12:02.118310 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:02.118697 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:02.254585 32645 raft_consensus.cc:993] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: : Instructing follower 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:02.255017 32645 raft_consensus.cc:1081] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Signalling peer 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:02.256644 32508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:48006
I20260709 13:12:02.257093 32508 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:02.257457 32508 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.261849 32508 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.263420 32508 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:12:02.264360 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:02.264442 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:02.265017 32358 raft_consensus.cc:3055] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:02.265075 32432 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.265352 32358 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:02.266106 32358 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.266999 32358 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.270965 32432 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:12:02.271412 32358 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:12:02.272192 32473 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:02.273077 32621 raft_consensus.cc:2804] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:02.274194 32621 raft_consensus.cc:697] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:02.274853 32621 consensus_queue.cc:237] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.281302 32256 catalog_manager.cc:5697] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 1 to 2, leader changed from 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1) to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.597632 32557 raft_consensus.cc:993] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:02.598146 32557 raft_consensus.cc:1081] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Signalling peer 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:02.599928 32508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:48008
I20260709 13:12:02.600626 32508 raft_consensus.cc:493] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:02.601001 32508 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.606082 32508 raft_consensus.cc:515] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.608068 32508 leader_election.cc:290] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:12:02.608886 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:02.609346 32432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:02.609479 32358 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.609926 32432 raft_consensus.cc:3055] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:02.610291 32432 raft_consensus.cc:740] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:02.611060 32432 consensus_queue.cc:260] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.612315 32432 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.616714 32358 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:12:02.617925 32474 leader_election.cc:304] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:02.618696 32621 raft_consensus.cc:2804] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:02.619135 32621 raft_consensus.cc:697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:02.619256 32432 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 2.
I20260709 13:12:02.619918 32621 consensus_queue.cc:237] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:02.629411 32256 catalog_manager.cc:5697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 1 to 2, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.771334 32358 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:02.772763 32621 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:02.784219 32432 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:02.785951 32658 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:03.119521 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:03.129580 32358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:50188
I20260709 13:12:03.130172 32358 raft_consensus.cc:606] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:12:03.135499 32508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:12:03.136034 32508 raft_consensus.cc:606] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:12:03.140913 32358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:50188
I20260709 13:12:03.141410 32358 raft_consensus.cc:606] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:12:03.145921 32358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:50188
I20260709 13:12:03.146379 32358 raft_consensus.cc:606] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:12:03.158317 32508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
mode: GRACEFUL
new_leader_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:12:03.158840 32508 raft_consensus.cc:606] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Received request to transfer leadership to TS 64724d62f48f482d8392d18f0c28441e
I20260709 13:12:03.225378 32432 raft_consensus.cc:1275] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.227005 32658 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.229642 32661 raft_consensus.cc:993] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.230165 32661 raft_consensus.cc:1081] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.231952 32434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59604
I20260709 13:12:03.232515 32434 raft_consensus.cc:493] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.232928 32434 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.233227 32657 raft_consensus.cc:993] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.233632 32657 raft_consensus.cc:1081] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.236892 32358 raft_consensus.cc:1275] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.237600 32433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59616
I20260709 13:12:03.238147 32433 raft_consensus.cc:493] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.238284 32657 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.238533 32433 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.248984 32433 raft_consensus.cc:515] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.251699 32359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:03.252480 32359 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.253351 32434 raft_consensus.cc:515] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.256641 32507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:03.256953 32433 leader_election.cc:290] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:03.257120 32507 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.257773 32434 leader_election.cc:290] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:03.260254 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c30c4c3dc7874a8d9cd18ce2f3130c46" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:03.260907 32508 raft_consensus.cc:3055] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:03.261231 32508 raft_consensus.cc:740] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:03.261857 32508 consensus_queue.cc:260] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.262719 32508 raft_consensus.cc:3060] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.262739 32507 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:12:03.263926 32399 leader_election.cc:304] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:03.264667 32557 raft_consensus.cc:2804] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.265710 32557 raft_consensus.cc:697] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:03.266490 32557 consensus_queue.cc:237] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.267369 32359 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.267897 32508 raft_consensus.cc:2468] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.268646 32400 leader_election.cc:304] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:03.269806 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:03.269835 32664 raft_consensus.cc:2804] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:03.270526 32358 raft_consensus.cc:3055] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.270879 32358 raft_consensus.cc:740] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:03.271561 32358 consensus_queue.cc:260] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.272503 32358 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.275728 32664 raft_consensus.cc:697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:03.276731 32664 consensus_queue.cc:237] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.277544 32358 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:12:03.280791 32253 catalog_manager.cc:5697] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 1 to 2, leader changed from 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 2 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.293704 32253 catalog_manager.cc:5697] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 2 to 3, leader changed from 282413e372c846f18cd8586ab1b610c0 (127.31.120.3) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.366873 32621 raft_consensus.cc:993] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.367360 32621 raft_consensus.cc:1081] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.368834 32434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59616
I20260709 13:12:03.369383 32434 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.369753 32434 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.375176 32434 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.376749 32645 raft_consensus.cc:993] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.377058 32434 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:03.377336 32645 raft_consensus.cc:1081] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.377972 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:03.378129 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:03.378541 32358 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.378700 32508 raft_consensus.cc:3055] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:03.379073 32508 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:03.379096 32434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59604
I20260709 13:12:03.379729 32434 raft_consensus.cc:493] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.380133 32434 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.379911 32508 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.380853 32508 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.384853 32508 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.385102 32358 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.385936 32399 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:03.386688 32664 raft_consensus.cc:2804] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:03.386663 32434 raft_consensus.cc:515] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.387269 32664 raft_consensus.cc:697] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:03.388078 32664 consensus_queue.cc:237] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.388691 32434 leader_election.cc:290] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:03.389585 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:03.390111 32358 raft_consensus.cc:3055] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:03.390108 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:03.390596 32358 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:03.390794 32508 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.391315 32358 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.392458 32358 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.396265 32508 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.395864 32253 catalog_manager.cc:5697] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 2 to 3, leader changed from 282413e372c846f18cd8586ab1b610c0 (127.31.120.3) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.396615 32358 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 3.
I20260709 13:12:03.397632 32400 leader_election.cc:304] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ebbeb35c22427db66f1ca4fa3730c5, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:03.398307 32664 raft_consensus.cc:2804] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:03.398768 32664 raft_consensus.cc:697] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:03.399554 32664 consensus_queue.cc:237] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.408465 32253 catalog_manager.cc:5697] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 2 to 3, leader changed from 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.465878 32661 raft_consensus.cc:993] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: : Instructing follower 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.466291 32661 raft_consensus.cc:1081] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Signalling peer 64724d62f48f482d8392d18f0c28441e to start an election
I20260709 13:12:03.467744 32434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
 from {username='slave'} at 127.0.0.1:59604
I20260709 13:12:03.468225 32434 raft_consensus.cc:493] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.468550 32434 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.472935 32434 raft_consensus.cc:515] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.474844 32434 leader_election.cc:290] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:03.475253 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:03.475862 32358 raft_consensus.cc:3055] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.475800 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "64724d62f48f482d8392d18f0c28441e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:03.476233 32358 raft_consensus.cc:740] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:03.476390 32508 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.476954 32358 consensus_queue.cc:260] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.478030 32358 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.482388 32508 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:12:03.483590 32358 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64724d62f48f482d8392d18f0c28441e in term 2.
I20260709 13:12:03.483543 32399 leader_election.cc:304] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 64724d62f48f482d8392d18f0c28441e; no voters: 
I20260709 13:12:03.484429 32664 raft_consensus.cc:2804] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.484825 32664 raft_consensus.cc:697] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Leader. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:03.485538 32664 consensus_queue.cc:237] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:03.491554 32253 catalog_manager.cc:5697] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e reported cstate change: term changed from 1 to 2, leader changed from 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1) to 64724d62f48f482d8392d18f0c28441e (127.31.120.2). New cstate: current_term: 2 leader_uuid: "64724d62f48f482d8392d18f0c28441e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.679672 32358 raft_consensus.cc:1275] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.680945 32664 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:03.688323 32508 raft_consensus.cc:1275] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.689777 32672 consensus_queue.cc:1048] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:03.749972 32508 raft_consensus.cc:1275] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.751113 32672 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.759346 32358 raft_consensus.cc:1275] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.760893 32557 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.808883 32508 raft_consensus.cc:1275] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.810432 32557 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:03.819680 32358 raft_consensus.cc:1275] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.821211 32664 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:03.832229 32508 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.833514 32664 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:03.842509 32358 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.844102 32664 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:03.874009 32358 raft_consensus.cc:1275] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.875432 32664 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.883848 32508 raft_consensus.cc:1275] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Refusing update from remote peer 64724d62f48f482d8392d18f0c28441e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.885727 32666 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:06.163746 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:06.171651 32434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:06.172418 32434 raft_consensus.cc:606] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Received request to transfer leadership to TS 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:12:06.175184 32434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:06.175707 32434 raft_consensus.cc:606] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Received request to transfer leadership to TS 282413e372c846f18cd8586ab1b610c0
I20260709 13:12:06.178125 32434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:06.178570 32434 raft_consensus.cc:606] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Received request to transfer leadership to TS 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:12:06.180900 32434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:06.181358 32434 raft_consensus.cc:606] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Received request to transfer leadership to TS 282413e372c846f18cd8586ab1b610c0
I20260709 13:12:06.183734 32434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "64724d62f48f482d8392d18f0c28441e"
mode: GRACEFUL
new_leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:59630
I20260709 13:12:06.184206 32434 raft_consensus.cc:606] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Received request to transfer leadership to TS 31ebbeb35c22427db66f1ca4fa3730c5
I20260709 13:12:06.185295 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:06.185760 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:06.228969 32557 raft_consensus.cc:993] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.229471 32666 raft_consensus.cc:1081] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Signalling peer 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.231084 32358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f95c9e5474f2449eb399be6862245506"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:50166
I20260709 13:12:06.231556 32358 raft_consensus.cc:493] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.231907 32358 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.236963 32358 raft_consensus.cc:515] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.238838 32358 leader_election.cc:290] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 4 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:06.239665 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:06.240038 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95c9e5474f2449eb399be6862245506" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:06.240394 32434 raft_consensus.cc:3055] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:06.240527 32508 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.240703 32434 raft_consensus.cc:740] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:06.241518 32434 consensus_queue.cc:260] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.242581 32434 raft_consensus.cc:3060] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.246214 32508 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 4.
I20260709 13:12:06.247959 32324 leader_election.cc:304] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:06.248272 32434 raft_consensus.cc:2468] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 4.
I20260709 13:12:06.249753 32682 raft_consensus.cc:2804] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:06.250536 32682 raft_consensus.cc:697] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:06.251312 32682 consensus_queue.cc:237] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.258940 32256 catalog_manager.cc:5697] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 3 to 4, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 4 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.328624 32666 raft_consensus.cc:993] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.329056 32666 raft_consensus.cc:1081] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Signalling peer 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.330524 32358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:50166
I20260709 13:12:06.330936 32358 raft_consensus.cc:493] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.331200 32358 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.335075 32358 raft_consensus.cc:515] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.336383 32358 leader_election.cc:290] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 3 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:06.337232 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:06.337499 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a50df29aed4877a0ba0933aa41df65" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:06.337800 32434 raft_consensus.cc:3055] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.337967 32508 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.338146 32434 raft_consensus.cc:740] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:06.338917 32434 consensus_queue.cc:260] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.340109 32434 raft_consensus.cc:3060] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.344071 32508 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 3.
I20260709 13:12:06.345098 32324 leader_election.cc:304] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:06.345889 32682 raft_consensus.cc:2804] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.346292 32682 raft_consensus.cc:697] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:06.346446 32434 raft_consensus.cc:2468] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 3.
I20260709 13:12:06.347170 32682 consensus_queue.cc:237] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.354238 32256 catalog_manager.cc:5697] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 2 to 3, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.469516 32557 raft_consensus.cc:993] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:06.469959 32666 raft_consensus.cc:1081] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Signalling peer 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:06.471424 32508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:48008
I20260709 13:12:06.471839 32508 raft_consensus.cc:493] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.472067 32508 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.476997 32508 raft_consensus.cc:515] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.478364 32508 leader_election.cc:290] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 3 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:12:06.479244 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:06.479491 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e558b1bf714079b1a62babadfb8468" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:06.479807 32358 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.480055 32434 raft_consensus.cc:3055] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.480367 32434 raft_consensus.cc:740] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:06.480988 32434 consensus_queue.cc:260] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.481851 32434 raft_consensus.cc:3060] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.483981 32358 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 3.
I20260709 13:12:06.485071 32474 leader_election.cc:304] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:06.485836 32621 raft_consensus.cc:2804] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.486361 32434 raft_consensus.cc:2468] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 3.
I20260709 13:12:06.487056 32621 raft_consensus.cc:697] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 3 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:06.487996 32621 consensus_queue.cc:237] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.495151 32256 catalog_manager.cc:5697] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 2 to 3, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.544821 32557 raft_consensus.cc:993] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.545266 32666 raft_consensus.cc:1081] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Signalling peer 31ebbeb35c22427db66f1ca4fa3730c5 to start an election
I20260709 13:12:06.546633 32358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300"
dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
 from {username='slave'} at 127.0.0.1:50166
I20260709 13:12:06.547034 32358 raft_consensus.cc:493] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.547356 32358 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.551407 32358 raft_consensus.cc:515] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.552826 32358 leader_election.cc:290] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 3 election: Requested vote from peers 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535), 282413e372c846f18cd8586ab1b610c0 (127.31.120.3:38427)
I20260709 13:12:06.553630 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:06.553881 32508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675e3147a0224eb5acf7c60137eab300" candidate_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "282413e372c846f18cd8586ab1b610c0"
I20260709 13:12:06.554243 32434 raft_consensus.cc:3055] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.554358 32508 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.554582 32434 raft_consensus.cc:740] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:06.555390 32434 consensus_queue.cc:260] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.556594 32434 raft_consensus.cc:3060] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.560708 32508 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 3.
I20260709 13:12:06.561786 32324 leader_election.cc:304] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:06.562491 32682 raft_consensus.cc:2804] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.562777 32434 raft_consensus.cc:2468] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ebbeb35c22427db66f1ca4fa3730c5 in term 3.
I20260709 13:12:06.562942 32682 raft_consensus.cc:697] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 LEADER]: Becoming Leader. State: Replica: 31ebbeb35c22427db66f1ca4fa3730c5, State: Running, Role: LEADER
I20260709 13:12:06.563915 32682 consensus_queue.cc:237] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.571408 32256 catalog_manager.cc:5697] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 reported cstate change: term changed from 2 to 3, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.643296 32557 raft_consensus.cc:993] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: : Instructing follower 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:06.643791 32666 raft_consensus.cc:1081] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Signalling peer 282413e372c846f18cd8586ab1b610c0 to start an election
I20260709 13:12:06.645239 32508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f"
dest_uuid: "282413e372c846f18cd8586ab1b610c0"
 from {username='slave'} at 127.0.0.1:48008
I20260709 13:12:06.645689 32508 raft_consensus.cc:493] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.645949 32508 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.649860 32508 raft_consensus.cc:515] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.651204 32508 leader_election.cc:290] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 4 election: Requested vote from peers 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931), 64724d62f48f482d8392d18f0c28441e (127.31.120.2:43535)
I20260709 13:12:06.652143 32358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "31ebbeb35c22427db66f1ca4fa3730c5"
I20260709 13:12:06.652204 32434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb7cbb23e5a42a8a6453b1d31043b9f" candidate_uuid: "282413e372c846f18cd8586ab1b610c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "64724d62f48f482d8392d18f0c28441e"
I20260709 13:12:06.652733 32434 raft_consensus.cc:3055] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:06.652750 32358 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.653113 32434 raft_consensus.cc:740] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64724d62f48f482d8392d18f0c28441e, State: Running, Role: LEADER
I20260709 13:12:06.653861 32434 consensus_queue.cc:260] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.654714 32434 raft_consensus.cc:3060] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.657379 32358 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 4.
I20260709 13:12:06.658592 32474 leader_election.cc:304] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282413e372c846f18cd8586ab1b610c0, 31ebbeb35c22427db66f1ca4fa3730c5; no voters: 
I20260709 13:12:06.659420 32621 raft_consensus.cc:2804] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:06.659933 32621 raft_consensus.cc:697] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 4 LEADER]: Becoming Leader. State: Replica: 282413e372c846f18cd8586ab1b610c0, State: Running, Role: LEADER
I20260709 13:12:06.660214 32434 raft_consensus.cc:2468] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282413e372c846f18cd8586ab1b610c0 in term 4.
I20260709 13:12:06.660851 32621 consensus_queue.cc:237] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } }
I20260709 13:12:06.668057 32256 catalog_manager.cc:5697] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 reported cstate change: term changed from 3 to 4, leader changed from 64724d62f48f482d8392d18f0c28441e (127.31.120.2) to 282413e372c846f18cd8586ab1b610c0 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "282413e372c846f18cd8586ab1b610c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.784495 32508 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:06.785658 32682 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:06.795699 32434 raft_consensus.cc:1275] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:06.797221 32696 consensus_queue.cc:1048] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:06.843024 32434 raft_consensus.cc:1275] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.844492 32696 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:06.850766 32508 raft_consensus.cc:1275] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.853070 32683 consensus_queue.cc:1048] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.014510 32358 raft_consensus.cc:1275] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.015779 32621 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ebbeb35c22427db66f1ca4fa3730c5" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.024817 32434 raft_consensus.cc:1275] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.026532 32621 consensus_queue.cc:1048] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.151793 32434 raft_consensus.cc:1275] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.153148 32701 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.163277 32507 raft_consensus.cc:1275] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Refusing update from remote peer 31ebbeb35c22427db66f1ca4fa3730c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.166941 32683 consensus_queue.cc:1048] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282413e372c846f18cd8586ab1b610c0" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.186517 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:07.191988 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:12:07.217727 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:07.218433 32224 tablet_replica.cc:333] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.219105 32224 raft_consensus.cc:2243] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.219650 32224 raft_consensus.cc:2272] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.221792 32224 tablet_replica.cc:333] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.222273 32224 raft_consensus.cc:2243] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.222687 32224 raft_consensus.cc:2272] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.224279 32224 tablet_replica.cc:333] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.224792 32224 raft_consensus.cc:2243] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.225207 32224 raft_consensus.cc:2272] T adb7cbb23e5a42a8a6453b1d31043b9f P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.226771 32224 tablet_replica.cc:333] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.227223 32224 raft_consensus.cc:2243] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.227955 32224 raft_consensus.cc:2272] T 675e3147a0224eb5acf7c60137eab300 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.229732 32224 tablet_replica.cc:333] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.230191 32224 raft_consensus.cc:2243] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.230841 32224 raft_consensus.cc:2272] T f95c9e5474f2449eb399be6862245506 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.232700 32224 tablet_replica.cc:333] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.233160 32224 raft_consensus.cc:2243] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.233536 32224 raft_consensus.cc:2272] T 71075fda0d764cfeb3d9bb99f121c3da P 31ebbeb35c22427db66f1ca4fa3730c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.235034 32224 tablet_replica.cc:333] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.235462 32224 raft_consensus.cc:2243] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.235855 32224 raft_consensus.cc:2272] T b49d887b268e41bea79aac32eac873ab P 31ebbeb35c22427db66f1ca4fa3730c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.237336 32224 tablet_replica.cc:333] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.237766 32224 raft_consensus.cc:2243] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.238431 32224 raft_consensus.cc:2272] T 03a50df29aed4877a0ba0933aa41df65 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.240589 32224 tablet_replica.cc:333] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5: stopping tablet replica
I20260709 13:12:07.241029 32224 raft_consensus.cc:2243] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.241420 32224 raft_consensus.cc:2272] T 03e558b1bf714079b1a62babadfb8468 P 31ebbeb35c22427db66f1ca4fa3730c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.243324 32434 raft_consensus.cc:1275] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Refusing update from remote peer 282413e372c846f18cd8586ab1b610c0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:07.244421 32621 consensus_queue.cc:1048] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64724d62f48f482d8392d18f0c28441e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:12:07.247673 32474 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:07.251466 32474 consensus_peers.cc:597] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 -> Peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Couldn't send request to peer 31ebbeb35c22427db66f1ca4fa3730c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:07.346741 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
I20260709 13:12:07.365996 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
I20260709 13:12:07.389617 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:07.390460 32224 tablet_replica.cc:333] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.391412 32224 raft_consensus.cc:2243] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.392278 32224 raft_consensus.cc:2272] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.393803 32224 tablet_replica.cc:333] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.394239 32224 raft_consensus.cc:2243] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.394562 32224 raft_consensus.cc:2272] T 675e3147a0224eb5acf7c60137eab300 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.396571 32224 tablet_replica.cc:333] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.396996 32224 raft_consensus.cc:2243] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.397305 32224 raft_consensus.cc:2272] T adb7cbb23e5a42a8a6453b1d31043b9f P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.399139 32224 tablet_replica.cc:333] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.399655 32224 raft_consensus.cc:2243] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.400270 32224 raft_consensus.cc:2272] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.401752 32224 tablet_replica.cc:333] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.402160 32224 raft_consensus.cc:2243] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.402468 32224 raft_consensus.cc:2272] T f95c9e5474f2449eb399be6862245506 P 64724d62f48f482d8392d18f0c28441e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.404448 32224 tablet_replica.cc:333] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.404865 32224 raft_consensus.cc:2243] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.405197 32224 raft_consensus.cc:2272] T 71075fda0d764cfeb3d9bb99f121c3da P 64724d62f48f482d8392d18f0c28441e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.406569 32224 tablet_replica.cc:333] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.406960 32224 raft_consensus.cc:2243] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.407548 32224 raft_consensus.cc:2272] T b49d887b268e41bea79aac32eac873ab P 64724d62f48f482d8392d18f0c28441e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.409313 32224 tablet_replica.cc:333] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.409703 32224 raft_consensus.cc:2243] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.410012 32224 raft_consensus.cc:2272] T 03e558b1bf714079b1a62babadfb8468 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.411818 32224 tablet_replica.cc:333] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e: stopping tablet replica
I20260709 13:12:07.412192 32224 raft_consensus.cc:2243] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.412497 32224 raft_consensus.cc:2272] T 03a50df29aed4877a0ba0933aa41df65 P 64724d62f48f482d8392d18f0c28441e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:07.426136 32474 consensus_peers.cc:597] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 -> Peer 31ebbeb35c22427db66f1ca4fa3730c5 (127.31.120.1:45931): Couldn't send request to peer 31ebbeb35c22427db66f1ca4fa3730c5. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:45931: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:07.441119 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
I20260709 13:12:07.457137 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:12:07.482719 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:07.483517 32224 tablet_replica.cc:333] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.484098 32224 raft_consensus.cc:2243] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.484562 32224 raft_consensus.cc:2272] T c30c4c3dc7874a8d9cd18ce2f3130c46 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.486326 32224 tablet_replica.cc:333] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.486860 32224 raft_consensus.cc:2243] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.487273 32224 raft_consensus.cc:2272] T 6b3b5f1d217a4782a3b6a3c322f1da77 P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.488934 32224 tablet_replica.cc:333] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.489428 32224 raft_consensus.cc:2243] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.490177 32224 raft_consensus.cc:2272] T adb7cbb23e5a42a8a6453b1d31043b9f P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.492218 32224 tablet_replica.cc:333] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.492766 32224 raft_consensus.cc:2243] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.493183 32224 raft_consensus.cc:2272] T 03a50df29aed4877a0ba0933aa41df65 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.495093 32224 tablet_replica.cc:333] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.495589 32224 raft_consensus.cc:2243] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.496407 32224 raft_consensus.cc:2272] T 03e558b1bf714079b1a62babadfb8468 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.498287 32224 tablet_replica.cc:333] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.499032 32224 raft_consensus.cc:2243] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.499822 32224 raft_consensus.cc:2272] T 71075fda0d764cfeb3d9bb99f121c3da P 282413e372c846f18cd8586ab1b610c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.501520 32224 tablet_replica.cc:333] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.501978 32224 raft_consensus.cc:2243] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.502346 32224 raft_consensus.cc:2272] T b49d887b268e41bea79aac32eac873ab P 282413e372c846f18cd8586ab1b610c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.504070 32224 tablet_replica.cc:333] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.504508 32224 raft_consensus.cc:2243] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.504916 32224 raft_consensus.cc:2272] T 675e3147a0224eb5acf7c60137eab300 P 282413e372c846f18cd8586ab1b610c0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.506834 32224 tablet_replica.cc:333] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0: stopping tablet replica
I20260709 13:12:07.507270 32224 raft_consensus.cc:2243] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:07.507745 32224 raft_consensus.cc:2272] T f95c9e5474f2449eb399be6862245506 P 282413e372c846f18cd8586ab1b610c0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.534219 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:12:07.551514 32224 master.cc:562] Master@127.31.120.62:42907 shutting down...
I20260709 13:12:07.664808 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:07.665357 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:07.665683 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P 95e7a9a3054a42cebacdd54e674e71ed: stopping tablet replica
I20260709 13:12:07.685652 32224 master.cc:584] Master@127.31.120.62:42907 shutdown complete.
282413e372c846f18cd8586ab1b610c0  1
31ebbeb35c22427db66f1ca4fa3730c5  4
64724d62f48f482d8392d18f0c28441e  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12499 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:12:07.726547 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:44129
I20260709 13:12:07.727802 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:07.733932 32713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:07.733986 32712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:07.734259 32715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:07.735276 32224 server_base.cc:1061] running on GCE node
I20260709 13:12:07.736109 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:07.736318 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:07.736476 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602727736445 us; error 0 us; skew 500 ppm
I20260709 13:12:07.736986 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:07.739392 32224 webserver.cc:533] Webserver started at http://127.31.120.62:35415/ using document root <none> and password file <none>
I20260709 13:12:07.740177 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:07.740374 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:07.740698 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:07.741804 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "b7131067716b49059f45890809976e34"
format_stamp: "Formatted at 2026-07-09 13:12:07 on dist-test-slave-pm98"
I20260709 13:12:07.746145 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:12:07.749411 32720 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:07.750057 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:12:07.750334 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "b7131067716b49059f45890809976e34"
format_stamp: "Formatted at 2026-07-09 13:12:07 on dist-test-slave-pm98"
I20260709 13:12:07.750672 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:07.773567 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:07.774814 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:07.812505 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:44129
I20260709 13:12:07.812597   303 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:44129 every 8 connection(s)
I20260709 13:12:07.816648   304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:07.825938   304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34: Bootstrap starting.
I20260709 13:12:07.830571   304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:07.834715   304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34: No bootstrap required, opened a new log
I20260709 13:12:07.836824   304 raft_consensus.cc:359] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER }
I20260709 13:12:07.837203   304 raft_consensus.cc:385] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:07.837486   304 raft_consensus.cc:740] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7131067716b49059f45890809976e34, State: Initialized, Role: FOLLOWER
I20260709 13:12:07.838094   304 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER }
I20260709 13:12:07.838542   304 raft_consensus.cc:399] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:07.838766   304 raft_consensus.cc:493] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:07.839044   304 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:07.842954   304 raft_consensus.cc:515] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER }
I20260709 13:12:07.843457   304 leader_election.cc:304] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7131067716b49059f45890809976e34; no voters: 
I20260709 13:12:07.844560   304 leader_election.cc:290] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:07.844885   307 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:07.846149   307 raft_consensus.cc:697] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 1 LEADER]: Becoming Leader. State: Replica: b7131067716b49059f45890809976e34, State: Running, Role: LEADER
I20260709 13:12:07.846803   307 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER }
I20260709 13:12:07.847393   304 sys_catalog.cc:565] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:07.851442   309 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7131067716b49059f45890809976e34. Latest consensus state: current_term: 1 leader_uuid: "b7131067716b49059f45890809976e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER } }
I20260709 13:12:07.851305   308 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7131067716b49059f45890809976e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7131067716b49059f45890809976e34" member_type: VOTER } }
I20260709 13:12:07.852159   309 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:07.852545   308 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:07.856523   313 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:07.864567   313 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:07.865522 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:07.873087   313 catalog_manager.cc:1383] Generated new cluster ID: d854cf2b2476454e9fdfdfaba3d8ff5d
I20260709 13:12:07.873387   313 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:07.893982   313 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:07.895217   313 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:07.915751   313 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34: Generated new TSK 0
I20260709 13:12:07.916491   313 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:07.932298 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:07.938925   325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:07.940500   326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:07.941340   328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:07.942011 32224 server_base.cc:1061] running on GCE node
I20260709 13:12:07.942981 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:07.943178 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:07.943382 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602727943365 us; error 0 us; skew 500 ppm
I20260709 13:12:07.944008 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:07.946476 32224 webserver.cc:533] Webserver started at http://127.31.120.1:44995/ using document root <none> and password file <none>
I20260709 13:12:07.947009 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:07.947227 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:07.947531 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:07.948791 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "937bbecd2c884fbab2e6b549bdc7be83"
format_stamp: "Formatted at 2026-07-09 13:12:07 on dist-test-slave-pm98"
I20260709 13:12:07.953747 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:07.957389   333 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:07.958159 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:12:07.958565 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "937bbecd2c884fbab2e6b549bdc7be83"
format_stamp: "Formatted at 2026-07-09 13:12:07 on dist-test-slave-pm98"
I20260709 13:12:07.958889 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:07.978391 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:07.979831 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:07.981482 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:07.983961 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:07.984210 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:07.984474 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:07.984675 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.027719 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:40979
I20260709 13:12:08.027807   395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:40979 every 8 connection(s)
I20260709 13:12:08.033051 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:08.041743   400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:08.043150   401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:08.044524 32224 server_base.cc:1061] running on GCE node
W20260709 13:12:08.045545   404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:08.046415 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:08.046638 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:08.046805 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602728046784 us; error 0 us; skew 500 ppm
I20260709 13:12:08.047298 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:08.050329 32224 webserver.cc:533] Webserver started at http://127.31.120.2:44083/ using document root <none> and password file <none>
I20260709 13:12:08.050990 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:08.051254 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:08.051304   396 heartbeater.cc:344] Connected to a master server at 127.31.120.62:44129
I20260709 13:12:08.051666 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:08.051730   396 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:08.052659   396 heartbeater.cc:507] Master 127.31.120.62:44129 requested a full tablet report, sending...
I20260709 13:12:08.053267 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "5d1d0e13ed034afca1c40284daf74edb"
format_stamp: "Formatted at 2026-07-09 13:12:08 on dist-test-slave-pm98"
I20260709 13:12:08.055043 32737 ts_manager.cc:194] Registered new tserver with Master: 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979)
I20260709 13:12:08.057495 32737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42038
I20260709 13:12:08.059289 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:12:08.062849   408 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.063561 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:08.063889 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "5d1d0e13ed034afca1c40284daf74edb"
format_stamp: "Formatted at 2026-07-09 13:12:08 on dist-test-slave-pm98"
I20260709 13:12:08.064172 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:08.083214 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:08.084410 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:08.085860 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:08.087924 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:08.088148 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.088377 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:08.088539 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.127535 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:32919
I20260709 13:12:08.127717   471 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:32919 every 8 connection(s)
I20260709 13:12:08.132114 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:08.140251   475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:08.140599   476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:08.143846 32224 server_base.cc:1061] running on GCE node
W20260709 13:12:08.144619   478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:08.145804   472 heartbeater.cc:344] Connected to a master server at 127.31.120.62:44129
I20260709 13:12:08.145874 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:08.146232 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:08.146246   472 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:08.146529 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602728146508 us; error 0 us; skew 500 ppm
I20260709 13:12:08.147114   472 heartbeater.cc:507] Master 127.31.120.62:44129 requested a full tablet report, sending...
I20260709 13:12:08.147204 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:08.149401 32737 ts_manager.cc:194] Registered new tserver with Master: 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:08.150223 32224 webserver.cc:533] Webserver started at http://127.31.120.3:41895/ using document root <none> and password file <none>
I20260709 13:12:08.150763 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:08.150918 32737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42040
I20260709 13:12:08.151033 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:08.151659 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:08.153225 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "16fce92c077847119c7e789058c25852"
format_stamp: "Formatted at 2026-07-09 13:12:08 on dist-test-slave-pm98"
I20260709 13:12:08.157903 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:08.161620   483 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.162406 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:08.162721 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "16fce92c077847119c7e789058c25852"
format_stamp: "Formatted at 2026-07-09 13:12:08 on dist-test-slave-pm98"
I20260709 13:12:08.162987 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:08.176767 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:08.177980 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:08.179397 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:08.181531 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:08.181749 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.181989 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:08.182158 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:08.221561 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:45563
I20260709 13:12:08.221675   545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:45563 every 8 connection(s)
I20260709 13:12:08.234997   546 heartbeater.cc:344] Connected to a master server at 127.31.120.62:44129
I20260709 13:12:08.235397   546 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:08.236110   546 heartbeater.cc:507] Master 127.31.120.62:44129 requested a full tablet report, sending...
I20260709 13:12:08.238132 32737 ts_manager.cc:194] Registered new tserver with Master: 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:08.239157 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014515989s
I20260709 13:12:08.239434 32737 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42044
I20260709 13:12:08.264070 32737 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42058:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:08.267164 32737 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:08.348189   361 tablet_service.cc:1511] Processing CreateTablet for tablet c407db8719844536a983858e0d68d43b (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:08.349992   361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c407db8719844536a983858e0d68d43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.349624   360 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee08ff22b14a249371c0a58f076903 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:08.350106   356 tablet_service.cc:1511] Processing CreateTablet for tablet 06834392e2b44bd58010de9e2debda51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:08.350230   358 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed5f57df654aeda85acfb967bfed51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:08.351519   360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee08ff22b14a249371c0a58f076903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.352140   356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06834392e2b44bd58010de9e2debda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.353777   357 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc17cd9ee3b4c95be367e78c1911fa5 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:08.355024   357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc17cd9ee3b4c95be367e78c1911fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.357448   359 tablet_service.cc:1511] Processing CreateTablet for tablet e289930f80f44defbb521f29ec049c92 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:08.358791   355 tablet_service.cc:1511] Processing CreateTablet for tablet f4e0bd8e52594d5f85bde9c940da47bf (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:08.360154   358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed5f57df654aeda85acfb967bfed51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.360811   359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e289930f80f44defbb521f29ec049c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.364542   355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e0bd8e52594d5f85bde9c940da47bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.370205   354 tablet_service.cc:1511] Processing CreateTablet for tablet c9eda5cd7fa34923b61f966013ceee21 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:08.371560   354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9eda5cd7fa34923b61f966013ceee21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.374573   436 tablet_service.cc:1511] Processing CreateTablet for tablet c407db8719844536a983858e0d68d43b (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:08.375660   436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c407db8719844536a983858e0d68d43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.377714   353 tablet_service.cc:1511] Processing CreateTablet for tablet e2497ebb53c7472292e77dc6e8c8bb6f (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:08.378969   353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2497ebb53c7472292e77dc6e8c8bb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.382300   352 tablet_service.cc:1511] Processing CreateTablet for tablet 55059494f790453b85cdf6ee41344040 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:08.383728   352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55059494f790453b85cdf6ee41344040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.387866   435 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee08ff22b14a249371c0a58f076903 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:08.389185   435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee08ff22b14a249371c0a58f076903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.406267   434 tablet_service.cc:1511] Processing CreateTablet for tablet e289930f80f44defbb521f29ec049c92 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:08.407976   430 tablet_service.cc:1511] Processing CreateTablet for tablet f4e0bd8e52594d5f85bde9c940da47bf (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:08.408787   433 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed5f57df654aeda85acfb967bfed51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:08.410547   429 tablet_service.cc:1511] Processing CreateTablet for tablet c9eda5cd7fa34923b61f966013ceee21 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:08.411046   432 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc17cd9ee3b4c95be367e78c1911fa5 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:08.413246   431 tablet_service.cc:1511] Processing CreateTablet for tablet 06834392e2b44bd58010de9e2debda51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:08.417210   428 tablet_service.cc:1511] Processing CreateTablet for tablet e2497ebb53c7472292e77dc6e8c8bb6f (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:08.413218   561 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.414772   434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e289930f80f44defbb521f29ec049c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.423811   511 tablet_service.cc:1511] Processing CreateTablet for tablet c407db8719844536a983858e0d68d43b (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:08.426906   508 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed5f57df654aeda85acfb967bfed51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:08.429407   509 tablet_service.cc:1511] Processing CreateTablet for tablet e289930f80f44defbb521f29ec049c92 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:08.425177   510 tablet_service.cc:1511] Processing CreateTablet for tablet 72ee08ff22b14a249371c0a58f076903 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:08.432996   431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06834392e2b44bd58010de9e2debda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.435663   506 tablet_service.cc:1511] Processing CreateTablet for tablet 06834392e2b44bd58010de9e2debda51 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:08.433528   507 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc17cd9ee3b4c95be367e78c1911fa5 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:08.438843   511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c407db8719844536a983858e0d68d43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.440419   505 tablet_service.cc:1511] Processing CreateTablet for tablet f4e0bd8e52594d5f85bde9c940da47bf (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:08.441978   504 tablet_service.cc:1511] Processing CreateTablet for tablet c9eda5cd7fa34923b61f966013ceee21 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:08.443701   507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc17cd9ee3b4c95be367e78c1911fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.433928   429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9eda5cd7fa34923b61f966013ceee21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.446020   427 tablet_service.cc:1511] Processing CreateTablet for tablet 55059494f790453b85cdf6ee41344040 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:08.448397   503 tablet_service.cc:1511] Processing CreateTablet for tablet e2497ebb53c7472292e77dc6e8c8bb6f (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:08.450352   506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06834392e2b44bd58010de9e2debda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.451025   503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2497ebb53c7472292e77dc6e8c8bb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.452014   508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed5f57df654aeda85acfb967bfed51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.456451   505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e0bd8e52594d5f85bde9c940da47bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.457192   504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9eda5cd7fa34923b61f966013ceee21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.459975   510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ee08ff22b14a249371c0a58f076903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.473321   502 tablet_service.cc:1511] Processing CreateTablet for tablet 55059494f790453b85cdf6ee41344040 (DEFAULT_TABLE table=test-workload [id=9807c6bad4b04787ac1600541915d66c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:08.475958   430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e0bd8e52594d5f85bde9c940da47bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.480077   433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed5f57df654aeda85acfb967bfed51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.480957   427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55059494f790453b85cdf6ee41344040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.481990   509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e289930f80f44defbb521f29ec049c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.482144   432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc17cd9ee3b4c95be367e78c1911fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.482872   502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55059494f790453b85cdf6ee41344040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.486765   428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2497ebb53c7472292e77dc6e8c8bb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:08.506641   561 tablet_bootstrap.cc:654] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.536087   563 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.536355   562 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.548049   562 tablet_bootstrap.cc:654] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.556262   563 tablet_bootstrap.cc:654] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.567286   561 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.567802   561 ts_tablet_manager.cc:1403] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.155s	user 0.010s	sys 0.005s
I20260709 13:12:08.572005   561 raft_consensus.cc:359] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.572724   561 raft_consensus.cc:385] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.573015   561 raft_consensus.cc:740] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.575362   561 consensus_queue.cc:260] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.579291   562 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:08.579842   562 ts_tablet_manager.cc:1403] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.006s
I20260709 13:12:08.583518   562 raft_consensus.cc:359] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.584218   562 raft_consensus.cc:385] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.584519   562 raft_consensus.cc:740] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.585384   562 consensus_queue.cc:260] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.588634   563 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:08.588955   472 heartbeater.cc:499] Master 127.31.120.62:44129 was elected leader, sending a full tablet report...
I20260709 13:12:08.589129   563 ts_tablet_manager.cc:1403] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.009s
I20260709 13:12:08.591363   396 heartbeater.cc:499] Master 127.31.120.62:44129 was elected leader, sending a full tablet report...
I20260709 13:12:08.592690   561 ts_tablet_manager.cc:1434] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.025s	user 0.000s	sys 0.007s
I20260709 13:12:08.593518   561 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.598325   561 tablet_bootstrap.cc:654] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.600299   562 ts_tablet_manager.cc:1434] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 13:12:08.601626   562 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.605149   563 raft_consensus.cc:359] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.607687   563 raft_consensus.cc:385] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.608708   563 raft_consensus.cc:740] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.609557   562 tablet_bootstrap.cc:654] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.609588   561 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.610306   561 ts_tablet_manager.cc:1403] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.011s
I20260709 13:12:08.609620   563 consensus_queue.cc:260] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.611758   567 raft_consensus.cc:493] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.612247   567 raft_consensus.cc:515] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.619217   567 leader_election.cc:290] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:08.626390   561 raft_consensus.cc:359] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.627418   561 raft_consensus.cc:385] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.636425   561 raft_consensus.cc:740] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.639981   561 consensus_queue.cc:260] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.648046   561 ts_tablet_manager.cc:1434] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:12:08.652611   568 raft_consensus.cc:493] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.653119   568 raft_consensus.cc:515] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.657963   561 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.667598   568 leader_election.cc:290] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:08.677794   563 ts_tablet_manager.cc:1434] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.088s	user 0.024s	sys 0.047s
I20260709 13:12:08.678948   447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
W20260709 13:12:08.681010   337 leader_election.cc:343] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.681860   546 heartbeater.cc:499] Master 127.31.120.62:44129 was elected leader, sending a full tablet report...
I20260709 13:12:08.686421   561 tablet_bootstrap.cc:654] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.687167   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
W20260709 13:12:08.688751   337 leader_election.cc:343] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.689210   337 leader_election.cc:304] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:08.690004   567 raft_consensus.cc:2749] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:08.691649   563 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.692554   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
W20260709 13:12:08.694254   410 leader_election.cc:343] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.698103   561 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.698640   563 tablet_bootstrap.cc:654] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.698736   561 ts_tablet_manager.cc:1403] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.006s
I20260709 13:12:08.701670   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
W20260709 13:12:08.703094   412 leader_election.cc:343] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.703553   412 leader_election.cc:304] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb; no voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:08.704032   562 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:08.704669   562 ts_tablet_manager.cc:1403] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.103s	user 0.038s	sys 0.014s
I20260709 13:12:08.704747   568 raft_consensus.cc:2749] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:08.701900   561 raft_consensus.cc:359] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.706931   561 raft_consensus.cc:385] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.707288   561 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.708534   563 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:08.708273   562 raft_consensus.cc:359] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:08.708356   561 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.709043   563 ts_tablet_manager.cc:1403] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.010s
I20260709 13:12:08.709086   562 raft_consensus.cc:385] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.709537   562 raft_consensus.cc:740] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.710376   562 consensus_queue.cc:260] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:08.712023   561 ts_tablet_manager.cc:1434] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 13:12:08.713058   561 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.716917   563 raft_consensus.cc:359] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.717828   563 raft_consensus.cc:385] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.718158   563 raft_consensus.cc:740] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.719063   563 consensus_queue.cc:260] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.720518   561 tablet_bootstrap.cc:654] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.721501   563 ts_tablet_manager.cc:1434] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:12:08.722498   563 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.724900   562 ts_tablet_manager.cc:1434] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:12:08.728299   562 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.730722   563 tablet_bootstrap.cc:654] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.735213   562 tablet_bootstrap.cc:654] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.736704   561 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.737135   561 ts_tablet_manager.cc:1403] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.012s
I20260709 13:12:08.739713   561 raft_consensus.cc:359] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.740248   562 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:08.740305   561 raft_consensus.cc:385] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.740252   563 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:08.740890   561 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.741029   562 ts_tablet_manager.cc:1403] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 13:12:08.741122   563 ts_tablet_manager.cc:1403] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 13:12:08.741722   561 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.743948   561 ts_tablet_manager.cc:1434] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:08.743974   563 raft_consensus.cc:359] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.744158   562 raft_consensus.cc:359] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.744936   563 raft_consensus.cc:385] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.745031   561 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.744989   562 raft_consensus.cc:385] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.745359   563 raft_consensus.cc:740] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.745473   562 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.746286   563 consensus_queue.cc:260] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.746356   562 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.748590   563 ts_tablet_manager.cc:1434] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:08.749567   563 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.751535   562 ts_tablet_manager.cc:1434] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:08.752174   561 tablet_bootstrap.cc:654] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.752584   562 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.756129   563 tablet_bootstrap.cc:654] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.760982   562 tablet_bootstrap.cc:654] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.767760   567 raft_consensus.cc:493] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.768479   567 raft_consensus.cc:515] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.772326   447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:08.773171   447 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 0.
I20260709 13:12:08.776135   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:08.778258   567 leader_election.cc:290] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:08.779109   337 leader_election.cc:304] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
W20260709 13:12:08.780615   337 leader_election.cc:343] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.791448   570 raft_consensus.cc:493] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.792173   570 raft_consensus.cc:515] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.803823   570 leader_election.cc:290] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:08.810303   567 raft_consensus.cc:2804] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:08.810981   567 raft_consensus.cc:493] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.811502   567 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:08.854360   567 raft_consensus.cc:515] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
W20260709 13:12:08.859454   317 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:08.869025   567 leader_election.cc:290] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:08.870154   570 raft_consensus.cc:493] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.870673   570 raft_consensus.cc:515] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.872397   562 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:08.872888   562 ts_tablet_manager.cc:1403] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.121s	user 0.041s	sys 0.064s
I20260709 13:12:08.872959   570 leader_election.cc:290] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:08.873751   561 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.873756   447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:08.874269   561 ts_tablet_manager.cc:1403] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.129s	user 0.037s	sys 0.088s
I20260709 13:12:08.874286   563 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:08.874500   447 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:08.874997   563 ts_tablet_manager.cc:1403] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.126s	user 0.045s	sys 0.071s
I20260709 13:12:08.875842   562 raft_consensus.cc:359] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.876540   562 raft_consensus.cc:385] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.876827   562 raft_consensus.cc:740] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.877352   561 raft_consensus.cc:359] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:08.877578   562 consensus_queue.cc:260] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.878072   561 raft_consensus.cc:385] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.877893   563 raft_consensus.cc:359] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.878418   561 raft_consensus.cc:740] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.878610   563 raft_consensus.cc:385] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.878944   563 raft_consensus.cc:740] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.878978   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:08.879187   561 consensus_queue.cc:260] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:08.879886   563 consensus_queue.cc:260] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.881548   561 ts_tablet_manager.cc:1434] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:08.882562   561 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:08.882385   563 ts_tablet_manager.cc:1434] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:12:08.884691   563 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.886787   562 ts_tablet_manager.cc:1434] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:12:08.889197   561 tablet_bootstrap.cc:654] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:08.890376   337 leader_election.cc:343] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.890918   563 tablet_bootstrap.cc:654] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.895169   562 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.904241   562 tablet_bootstrap.cc:654] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.915349   447 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 1.
I20260709 13:12:08.922039   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:08.928447   446 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:08.923990   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:08.924346   562 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:08.930409   562 ts_tablet_manager.cc:1403] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.008s
I20260709 13:12:08.930202   487 leader_election.cc:304] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
W20260709 13:12:08.931278   487 leader_election.cc:343] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.931990   570 raft_consensus.cc:2804] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:08.932487   561 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:08.932971   561 ts_tablet_manager.cc:1403] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.051s	user 0.029s	sys 0.020s
I20260709 13:12:08.933014   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:08.933068   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:08.932360   570 raft_consensus.cc:493] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.933455   562 raft_consensus.cc:359] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.934198   570 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:08.931181   337 leader_election.cc:304] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852
I20260709 13:12:08.934392   562 raft_consensus.cc:385] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.934784   562 raft_consensus.cc:740] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.935302   567 raft_consensus.cc:2804] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:12:08.936484   485 leader_election.cc:343] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.936940   561 raft_consensus.cc:359] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.937611   561 raft_consensus.cc:385] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:08.937426   485 leader_election.cc:343] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:08.937960   561 raft_consensus.cc:740] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.938552   485 leader_election.cc:304] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16fce92c077847119c7e789058c25852; no voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:08.938956   561 consensus_queue.cc:260] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.935794   562 consensus_queue.cc:260] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.944775   563 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:08.945331   563 ts_tablet_manager.cc:1403] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.024s
I20260709 13:12:08.947464   562 ts_tablet_manager.cc:1434] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:08.951581   563 raft_consensus.cc:359] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.952513   563 raft_consensus.cc:385] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:08.952857   563 raft_consensus.cc:740] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:08.950488   570 raft_consensus.cc:515] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.954483   562 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:08.956240   570 leader_election.cc:290] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:08.956466   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:08.956908   570 raft_consensus.cc:2749] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:08.957099   445 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:08.957814   563 consensus_queue.cc:260] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.963435   563 ts_tablet_manager.cc:1434] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:12:08.964884   563 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:08.966197   567 raft_consensus.cc:697] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:08.967530   567 consensus_queue.cc:237] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.971210   590 raft_consensus.cc:493] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.973742   590 raft_consensus.cc:515] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:08.972291   563 tablet_bootstrap.cc:654] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:08.997224   445 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.000486   562 tablet_bootstrap.cc:654] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.004367   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.004823   561 ts_tablet_manager.cc:1434] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.071s	user 0.019s	sys 0.048s
I20260709 13:12:09.005129   521 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 0.
I20260709 13:12:09.003165   590 leader_election.cc:290] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.004381   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.012073   337 leader_election.cc:304] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.013049   567 raft_consensus.cc:2804] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:09.013195   337 leader_election.cc:343] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.013715   567 raft_consensus.cc:493] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.013237 32737 catalog_manager.cc:5697] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.014329   567 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.014446   487 leader_election.cc:304] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:09.016063   593 raft_consensus.cc:2804] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.016716   593 raft_consensus.cc:697] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.017846   593 consensus_queue.cc:237] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.014623   590 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.019438   590 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.021498   561 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
W20260709 13:12:09.000486   485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:40979, user_credentials={real_user=slave}} blocked reactor thread for 63055us
I20260709 13:12:09.024812   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.030074   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
W20260709 13:12:09.031966   337 leader_election.cc:343] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.034389   590 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
W20260709 13:12:09.032876   485 leader_election.cc:343] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.038360   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.039841   563 tablet_bootstrap.cc:492] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:09.039996   561 tablet_bootstrap.cc:654] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.040347   563 ts_tablet_manager.cc:1403] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.015s
W20260709 13:12:09.043327   337 leader_election.cc:343] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.043919   337 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.045641   563 raft_consensus.cc:359] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:09.046465   590 raft_consensus.cc:2749] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.045346 32735 catalog_manager.cc:5697] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.047192   563 raft_consensus.cc:385] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.049243   563 raft_consensus.cc:740] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.049885   567 raft_consensus.cc:515] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.050062   563 consensus_queue.cc:260] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:09.051998   567 leader_election.cc:290] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.052769   563 ts_tablet_manager.cc:1434] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.005s
I20260709 13:12:09.053782   563 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:09.054508   561 tablet_bootstrap.cc:492] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:09.054205   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.054978   561 ts_tablet_manager.cc:1403] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.003s
I20260709 13:12:09.056329   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.056915   521 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.058293   561 raft_consensus.cc:359] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.058251   562 tablet_bootstrap.cc:492] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
W20260709 13:12:09.059638   337 leader_election.cc:343] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.060596   563 tablet_bootstrap.cc:654] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.059100   561 raft_consensus.cc:385] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.061645   561 raft_consensus.cc:740] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.061882   567 raft_consensus.cc:493] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.062363   567 raft_consensus.cc:515] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:09.062423   561 consensus_queue.cc:260] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.066078   561 ts_tablet_manager.cc:1434] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 13:12:09.059762   562 ts_tablet_manager.cc:1403] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.011s
I20260709 13:12:09.067029   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.068559   520 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 0.
I20260709 13:12:09.069121   561 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:09.069823   337 leader_election.cc:304] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.067034   567 leader_election.cc:290] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16fce92c077847119c7e789058c25852 (127.31.120.3:45563), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.069895   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.070765   567 raft_consensus.cc:2804] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.071136   567 raft_consensus.cc:493] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.071017   446 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 0.
I20260709 13:12:09.071524   567 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.074743   562 raft_consensus.cc:359] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.078022   562 raft_consensus.cc:385] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.078574   562 raft_consensus.cc:740] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.083113   521 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 1.
I20260709 13:12:09.084951   337 leader_election.cc:304] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.086457   590 raft_consensus.cc:2804] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.087189   590 raft_consensus.cc:697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.087082   562 consensus_queue.cc:260] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.087169   561 tablet_bootstrap.cc:654] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.088387   590 consensus_queue.cc:237] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.089040   567 raft_consensus.cc:515] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:09.091814   562 ts_tablet_manager.cc:1434] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 13:12:09.092815   562 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:09.094738   593 raft_consensus.cc:493] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.095217   593 raft_consensus.cc:515] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.096259   563 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:09.096727   563 ts_tablet_manager.cc:1403] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.015s
I20260709 13:12:09.097414   593 leader_election.cc:290] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.098989   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.100247   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.100415   567 leader_election.cc:290] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Requested vote from peers 16fce92c077847119c7e789058c25852 (127.31.120.3:45563), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.102998   593 raft_consensus.cc:493] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.103494   593 raft_consensus.cc:515] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
W20260709 13:12:09.103936   485 leader_election.cc:343] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:09.105396   487 leader_election.cc:343] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.105837   487 leader_election.cc:304] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16fce92c077847119c7e789058c25852; no voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.106457   593 leader_election.cc:290] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.106930   593 raft_consensus.cc:2749] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.108202   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.108310   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.108837   446 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.108918   445 raft_consensus.cc:2468] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:09.109164   562 tablet_bootstrap.cc:654] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.110086   568 raft_consensus.cc:493] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.111591   487 leader_election.cc:304] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:09.112321   568 raft_consensus.cc:515] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.112823   570 raft_consensus.cc:2804] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.113200   570 raft_consensus.cc:493] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.113523   570 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.114372   568 leader_election.cc:290] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.116034   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.117007   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.117225   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.117983   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
W20260709 13:12:09.118427   410 leader_election.cc:343] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.118755   521 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:09.119259   412 leader_election.cc:343] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16fce92c077847119c7e789058c25852 (127.31.120.3:45563): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:09.119457   485 leader_election.cc:343] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.119724   412 leader_election.cc:304] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb; no voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.121182   568 raft_consensus.cc:2749] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.126920   561 tablet_bootstrap.cc:492] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:09.127394   561 ts_tablet_manager.cc:1403] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.009s
I20260709 13:12:09.128114   570 raft_consensus.cc:515] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.131778   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.132555   563 raft_consensus.cc:359] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.133155   563 raft_consensus.cc:385] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.133118   561 raft_consensus.cc:359] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.133576   563 raft_consensus.cc:740] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.133882   561 raft_consensus.cc:385] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:09.133975   485 leader_election.cc:343] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:09.131778   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.134214   561 raft_consensus.cc:740] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.134586   570 leader_election.cc:290] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.134766   445 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.135008   561 consensus_queue.cc:260] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.135249   563 consensus_queue.cc:260] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.137025   561 ts_tablet_manager.cc:1434] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:12:09.137490   563 ts_tablet_manager.cc:1434] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:12:09.137940   561 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:09.138481   563 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:09.141649   445 raft_consensus.cc:2468] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.141224 32735 catalog_manager.cc:5697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.142866   487 leader_election.cc:304] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.144402   561 tablet_bootstrap.cc:654] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.145326   570 raft_consensus.cc:2804] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.145850   570 raft_consensus.cc:697] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.145980   446 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 1.
I20260709 13:12:09.146705   570 consensus_queue.cc:237] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.147034   337 leader_election.cc:304] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.147807   567 raft_consensus.cc:2804] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.148276   567 raft_consensus.cc:697] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.149035   567 consensus_queue.cc:237] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:09.163724   521 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 1.
I20260709 13:12:09.164866   562 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:09.165325   562 ts_tablet_manager.cc:1403] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.001s
I20260709 13:12:09.164088   563 tablet_bootstrap.cc:654] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.168253   562 raft_consensus.cc:359] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.170990   562 raft_consensus.cc:385] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.171325   562 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.172281   562 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.179527 32737 catalog_manager.cc:5697] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.181811   561 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:09.182408   561 ts_tablet_manager.cc:1403] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.013s
I20260709 13:12:09.183115   562 ts_tablet_manager.cc:1434] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:09.185719   561 raft_consensus.cc:359] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.186546   561 raft_consensus.cc:385] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.186796   561 raft_consensus.cc:740] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.187441   561 consensus_queue.cc:260] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.188301   562 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:09.189688   561 ts_tablet_manager.cc:1434] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:09.189625 32735 catalog_manager.cc:5697] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.190641   561 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: Bootstrap starting.
I20260709 13:12:09.193634   562 tablet_bootstrap.cc:654] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.196471   561 tablet_bootstrap.cc:654] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.205413   562 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:09.205862   562 ts_tablet_manager.cc:1403] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.001s
I20260709 13:12:09.207759   563 tablet_bootstrap.cc:492] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:09.208182   563 ts_tablet_manager.cc:1403] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.003s
I20260709 13:12:09.208834   562 raft_consensus.cc:359] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.209489   562 raft_consensus.cc:385] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.209757   562 raft_consensus.cc:740] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.210399   562 consensus_queue.cc:260] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.210556   563 raft_consensus.cc:359] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.211118   563 raft_consensus.cc:385] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.211397   563 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.211833   561 tablet_bootstrap.cc:492] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: No bootstrap required, opened a new log
I20260709 13:12:09.212237   561 ts_tablet_manager.cc:1403] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:12:09.212059   563 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.214082   563 ts_tablet_manager.cc:1434] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:09.214578   561 raft_consensus.cc:359] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.216408   561 raft_consensus.cc:385] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.216675   561 raft_consensus.cc:740] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.216766   563 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:09.215291   562 ts_tablet_manager.cc:1434] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:12:09.218062   562 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:09.223414   562 tablet_bootstrap.cc:654] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.223747   563 tablet_bootstrap.cc:654] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.224860   561 consensus_queue.cc:260] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.226833   561 ts_tablet_manager.cc:1434] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:12:09.228485   562 tablet_bootstrap.cc:492] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:09.228984   562 ts_tablet_manager.cc:1403] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 13:12:09.230031   563 tablet_bootstrap.cc:492] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:09.230446   563 ts_tablet_manager.cc:1403] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:12:09.231594   562 raft_consensus.cc:359] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.232311   562 raft_consensus.cc:385] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.232626   562 raft_consensus.cc:740] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.232888   563 raft_consensus.cc:359] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.233494   563 raft_consensus.cc:385] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.233886   563 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.233474   562 consensus_queue.cc:260] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.234524   563 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.236260   562 ts_tablet_manager.cc:1434] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:09.237121   562 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: Bootstrap starting.
I20260709 13:12:09.239697   563 ts_tablet_manager.cc:1434] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:12:09.240634   563 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: Bootstrap starting.
I20260709 13:12:09.242859   562 tablet_bootstrap.cc:654] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.245010   570 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.245463   570 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.246832   563 tablet_bootstrap.cc:654] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:09.248562   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.249193   446 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:09.249631   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.250381   371 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:09.250762   487 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:09.247462   570 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.251523   593 raft_consensus.cc:2804] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.251922   593 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.252249   593 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.253044   563 tablet_bootstrap.cc:492] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: No bootstrap required, opened a new log
I20260709 13:12:09.253466   563 ts_tablet_manager.cc:1403] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: Time spent bootstrapping tablet: real 0.013s	user 0.003s	sys 0.008s
I20260709 13:12:09.256044   563 raft_consensus.cc:359] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.256536   568 raft_consensus.cc:493] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.256688   563 raft_consensus.cc:385] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.256983   563 raft_consensus.cc:740] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.256989   568 raft_consensus.cc:515] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.257745   563 consensus_queue.cc:260] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.259075   568 leader_election.cc:290] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.259889   563 ts_tablet_manager.cc:1434] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:09.260775   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.261399   371 raft_consensus.cc:2468] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.261749   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.259878   593 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.262401   521 raft_consensus.cc:2468] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.262566   410 leader_election.cc:304] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.263485   568 raft_consensus.cc:2804] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.263871   568 raft_consensus.cc:493] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.264194   568 raft_consensus.cc:3060] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.264505   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.265054   371 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.268950   593 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.271369   568 raft_consensus.cc:515] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.271842   371 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.273102   485 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.273958   593 raft_consensus.cc:2804] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.274188   568 leader_election.cc:290] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.274962   568 raft_consensus.cc:493] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.275532   562 tablet_bootstrap.cc:492] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: No bootstrap required, opened a new log
I20260709 13:12:09.275717   593 raft_consensus.cc:697] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.276876   562 ts_tablet_manager.cc:1403] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.009s
I20260709 13:12:09.275475   568 raft_consensus.cc:515] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.277562   593 consensus_queue.cc:237] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.279403   562 raft_consensus.cc:359] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.279805   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.279928   568 leader_election.cc:290] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.280182   562 raft_consensus.cc:385] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:09.280447   370 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.280519   562 raft_consensus.cc:740] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Initialized, Role: FOLLOWER
I20260709 13:12:09.276093   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.281352   371 raft_consensus.cc:3060] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.281471   562 consensus_queue.cc:260] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.281876   410 leader_election.cc:304] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.282758   568 raft_consensus.cc:2804] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.283239   568 raft_consensus.cc:493] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.283771   562 ts_tablet_manager.cc:1434] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:09.283697   568 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.287794   371 raft_consensus.cc:2468] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.292136   410 leader_election.cc:304] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.292846   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.293049   617 raft_consensus.cc:2804] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.293462   446 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.293473 32736 catalog_manager.cc:5697] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.300503   446 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.302251   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2497ebb53c7472292e77dc6e8c8bb6f" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.302837   521 raft_consensus.cc:3060] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.310698   568 raft_consensus.cc:515] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.312811   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.313503   520 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.314072   568 leader_election.cc:290] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.314110   617 raft_consensus.cc:697] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:09.315400   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.315265   617 consensus_queue.cc:237] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.313519   620 raft_consensus.cc:493] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.316015   371 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.316406   620 raft_consensus.cc:515] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.319187   370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.319626   620 leader_election.cc:290] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.319718   370 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.319991   519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.320429   521 raft_consensus.cc:2468] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.320606   519 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 0.
I20260709 13:12:09.321419   410 leader_election.cc:304] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.322238   620 raft_consensus.cc:2804] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.322592   620 raft_consensus.cc:493] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.322428   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.322907   620 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.323261   520 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.327736   371 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.329157   410 leader_election.cc:304] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.330332   619 raft_consensus.cc:2804] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.330958   619 raft_consensus.cc:697] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:09.332082   619 consensus_queue.cc:237] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.334517 32736 catalog_manager.cc:5697] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 1 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.332306   520 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.337146   620 raft_consensus.cc:515] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.339336   620 leader_election.cc:290] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.339574   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.340199   371 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.340378   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.340969   520 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.345633   371 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.346554   410 leader_election.cc:304] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.347183   620 raft_consensus.cc:2804] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.347247   520 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 1.
I20260709 13:12:09.347668   620 raft_consensus.cc:697] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:09.348628   619 raft_consensus.cc:493] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.348486   620 consensus_queue.cc:237] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.349062   619 raft_consensus.cc:515] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.351034   619 leader_election.cc:290] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.350678 32736 catalog_manager.cc:5697] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 1 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.352214   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.352568   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.353138   520 raft_consensus.cc:2393] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d1d0e13ed034afca1c40284daf74edb in current term 1: Already voted for candidate 937bbecd2c884fbab2e6b549bdc7be83 in this term.
I20260709 13:12:09.354558   410 leader_election.cc:304] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb; no voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.355417   620 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.361930   620 raft_consensus.cc:2749] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.364111   520 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.365274 32736 catalog_manager.cc:5697] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 1 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.370251   567 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.383407   567 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:09.386067   567 raft_consensus.cc:493] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.388684   567 raft_consensus.cc:515] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.390512   567 leader_election.cc:290] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.391233   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.391467   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.392378   445 raft_consensus.cc:2393] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 937bbecd2c884fbab2e6b549bdc7be83 in current term 1: Already voted for candidate 16fce92c077847119c7e789058c25852 in this term.
I20260709 13:12:09.393452   337 leader_election.cc:304] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.394124   567 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.401113   567 raft_consensus.cc:2749] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.420096   567 raft_consensus.cc:493] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.420621   567 raft_consensus.cc:515] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.422556   567 leader_election.cc:290] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.423065   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.423555   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16fce92c077847119c7e789058c25852" is_pre_election: true
I20260709 13:12:09.423811   446 raft_consensus.cc:2393] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 937bbecd2c884fbab2e6b549bdc7be83 in current term 1: Already voted for candidate 16fce92c077847119c7e789058c25852 in this term.
I20260709 13:12:09.425022   337 leader_election.cc:304] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.425642   567 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.430361   567 raft_consensus.cc:2749] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:09.431932   593 raft_consensus.cc:493] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.432331   593 raft_consensus.cc:515] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.434008   593 leader_election.cc:290] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.434950   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83" is_pre_election: true
I20260709 13:12:09.434945   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb" is_pre_election: true
I20260709 13:12:09.435592   371 raft_consensus.cc:2468] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:09.435762   446 raft_consensus.cc:2468] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 0.
I20260709 13:12:09.436641   485 leader_election.cc:304] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.437287   593 raft_consensus.cc:2804] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:09.437628   593 raft_consensus.cc:493] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.437866   593 raft_consensus.cc:3060] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.442499   593 raft_consensus.cc:515] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.444289   593 leader_election.cc:290] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:09.445029   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.445307   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55059494f790453b85cdf6ee41344040" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.445520   371 raft_consensus.cc:3060] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.445871   446 raft_consensus.cc:3060] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:09.449994   371 raft_consensus.cc:2468] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.450547   446 raft_consensus.cc:2468] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 1.
I20260709 13:12:09.450984   485 leader_election.cc:304] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.451761   593 raft_consensus.cc:2804] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:09.452272   593 raft_consensus.cc:697] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.452993   593 consensus_queue.cc:237] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.460436 32735 catalog_manager.cc:5697] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.486065   520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903"
dest_uuid: "16fce92c077847119c7e789058c25852"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40360
I20260709 13:12:09.486626   520 raft_consensus.cc:606] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.491586   520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "16fce92c077847119c7e789058c25852"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40360
I20260709 13:12:09.492177   520 raft_consensus.cc:606] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.494997   593 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.510310   445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40468
I20260709 13:12:09.510931   445 raft_consensus.cc:606] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:09.512470   634 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:09.530155   371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:38776
I20260709 13:12:09.530774   371 raft_consensus.cc:606] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Received request to transfer leadership to TS 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.531354   634 consensus_queue.cc:1048] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.536285   567 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.545545   570 consensus_queue.cc:1048] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.545979   521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21"
dest_uuid: "16fce92c077847119c7e789058c25852"
mode: GRACEFUL
new_leader_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:40360
I20260709 13:12:09.546746   521 raft_consensus.cc:606] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Received request to transfer leadership to TS 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:09.563680   590 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:09.566751   593 raft_consensus.cc:993] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.567174   634 raft_consensus.cc:1081] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.569154   371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38772
I20260709 13:12:09.569674   371 raft_consensus.cc:493] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:09.570062   371 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.576928   371 raft_consensus.cc:515] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.578846   371 leader_election.cc:290] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.579886   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.580454   445 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.580904   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ee08ff22b14a249371c0a58f076903" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.581547   520 raft_consensus.cc:3055] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:09.581921   520 raft_consensus.cc:740] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.582916   520 consensus_queue.cc:260] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.584286   520 raft_consensus.cc:3060] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.587324   445 raft_consensus.cc:2468] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:09.588465   337 leader_election.cc:304] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.589478   640 raft_consensus.cc:2804] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:09.590274   640 raft_consensus.cc:697] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.591244   640 consensus_queue.cc:237] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.595510   520 raft_consensus.cc:2468] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:09.599100 32737 catalog_manager.cc:5697] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 1 to 2, leader changed from 16fce92c077847119c7e789058c25852 (127.31.120.3) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.648763   640 consensus_queue.cc:1048] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.660215   590 consensus_queue.cc:1048] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.678510   634 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.689364   593 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:09.697948   593 raft_consensus.cc:993] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.698411   570 raft_consensus.cc:1081] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.699985   371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38772
I20260709 13:12:09.700552   371 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:09.701051   371 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.706072   371 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.707813   371 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.708664   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.709231   445 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.709079   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.709687   520 raft_consensus.cc:3055] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:09.710011   520 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:09.710793   520 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.711894   520 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.715318   445 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:09.716423   337 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:09.716744   520 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:09.717134   590 raft_consensus.cc:2804] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:09.717584   590 raft_consensus.cc:697] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.718302   590 consensus_queue.cc:237] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.724363 32737 catalog_manager.cc:5697] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 1 to 2, leader changed from 16fce92c077847119c7e789058c25852 (127.31.120.3) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.733887   620 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.744077   568 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.774727   640 raft_consensus.cc:993] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:09.775214   590 raft_consensus.cc:1081] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Signalling peer 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:09.776754   445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:40450
I20260709 13:12:09.777237   445 raft_consensus.cc:493] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:09.777541   445 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.782045   445 raft_consensus.cc:515] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.783706   445 leader_election.cc:290] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.784713   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.784864   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:09.785334   520 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.785380   371 raft_consensus.cc:3055] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:09.785753   371 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.786424   371 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.787287   371 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.790264   520 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:09.791214   412 leader_election.cc:304] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:09.791923   371 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:09.791937   568 raft_consensus.cc:2804] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:09.792634   568 raft_consensus.cc:697] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:09.793356   568 consensus_queue.cc:237] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.799721 32736 catalog_manager.cc:5697] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 1 to 2, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 2 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.805518   568 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.812299   619 raft_consensus.cc:993] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.812758   617 raft_consensus.cc:1081] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:09.813052   620 consensus_queue.cc:1048] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.814451   371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38770
I20260709 13:12:09.814997   371 raft_consensus.cc:493] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:09.815382   371 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.818672   620 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.826683   371 raft_consensus.cc:515] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.827984   370 raft_consensus.cc:1240] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Rejecting Update request from peer 5d1d0e13ed034afca1c40284daf74edb for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:09.833389   620 consensus_queue.cc:1059] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:09.834640   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:09.835126   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:09.835419   619 consensus_queue.cc:1048] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.836141   371 leader_election.cc:290] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:09.845691   445 raft_consensus.cc:3055] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:09.845988   445 raft_consensus.cc:740] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:09.846658   445 consensus_queue.cc:260] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.847685   445 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:09.853807   445 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
W20260709 13:12:09.854269   619 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:12:09.855965   337 leader_election.cc:304] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 16fce92c077847119c7e789058c25852
I20260709 13:12:09.856818   590 raft_consensus.cc:2804] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:09.857331   590 raft_consensus.cc:697] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:09.858104   590 consensus_queue.cc:237] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:09.864410 32736 catalog_manager.cc:5697] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 1 to 2, leader changed from 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:09.958379   570 consensus_queue.cc:1048] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:09.968581   570 consensus_queue.cc:1048] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:10.046547   445 raft_consensus.cc:1275] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.047742   590 consensus_queue.cc:1048] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.055217   521 raft_consensus.cc:1275] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.056623   640 consensus_queue.cc:1048] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.065047   634 raft_consensus.cc:993] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: : Instructing follower 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:10.065513   634 raft_consensus.cc:1081] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Signalling peer 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:10.067018   445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:12:10.067529   445 raft_consensus.cc:493] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.067839   445 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.074203   445 raft_consensus.cc:515] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:10.076134   445 leader_election.cc:290] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:10.077138   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:10.077397   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:10.077734   371 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.077920   521 raft_consensus.cc:3055] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:10.078299   521 raft_consensus.cc:740] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:10.079017   521 consensus_queue.cc:260] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:10.080102   521 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.082899   371 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:10.083945   410 leader_election.cc:304] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:10.084619   619 raft_consensus.cc:2804] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:10.085052   619 raft_consensus.cc:697] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:10.085494   521 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:10.085829   619 consensus_queue.cc:237] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:10.093072 32735 catalog_manager.cc:5697] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 1 to 2, leader changed from 16fce92c077847119c7e789058c25852 (127.31.120.3) to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 2 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.182389   445 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.183629   590 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.190649   521 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.192065   640 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.367132   521 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.368373   619 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.376116   371 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.378134   654 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.418846   521 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.422335   445 raft_consensus.cc:1275] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.423828   592 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.424450   521 raft_consensus.cc:1275] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.425666   592 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.486949   521 raft_consensus.cc:1275] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.488344   654 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.496802   371 raft_consensus.cc:1275] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.498194   568 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:12.593431 32224 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:12:12.594143 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:12.610546   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:12.611092   369 raft_consensus.cc:606] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Received request to transfer leadership to TS 16fce92c077847119c7e789058c25852
I20260709 13:12:12.613870   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:12.614454   369 raft_consensus.cc:606] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Received request to transfer leadership to TS 16fce92c077847119c7e789058c25852
I20260709 13:12:12.615512 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:12.615914 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:12.670346   657 raft_consensus.cc:993] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:12.670759   590 raft_consensus.cc:1081] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Signalling peer 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:12.672279   521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:40346
I20260709 13:12:12.672716   521 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.672963   521 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:12.678063   521 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.679804   521 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 3 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:12.680807   369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:12.680905   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:12.681488   369 raft_consensus.cc:3055] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:12.681710   445 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:12.681906   369 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:12.682803   369 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.683815   369 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:12.686606   445 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 3.
I20260709 13:12:12.687668   487 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:12.688370   634 raft_consensus.cc:2804] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:12.688923   369 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 3.
I20260709 13:12:12.689464   634 raft_consensus.cc:697] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:12.690351   634 consensus_queue.cc:237] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.697631 32735 catalog_manager.cc:5697] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 2 to 3, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:12.820106   657 raft_consensus.cc:993] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:12.820595   590 raft_consensus.cc:1081] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Signalling peer 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:12.822115   521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5"
dest_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:40346
I20260709 13:12:12.822666   521 raft_consensus.cc:493] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.823001   521 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.827910   521 raft_consensus.cc:515] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.829689   521 leader_election.cc:290] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 2 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:12.830580   371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:12.830659   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:12.831391   445 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.831352   371 raft_consensus.cc:3055] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:12.831926   371 raft_consensus.cc:740] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:12.832581   371 consensus_queue.cc:260] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.833463   371 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.835755   445 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 2.
I20260709 13:12:12.836694   487 leader_election.cc:304] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:12.837532   660 raft_consensus.cc:2804] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:12.837946   660 raft_consensus.cc:697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:12.838301   371 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 2.
I20260709 13:12:12.838670   660 consensus_queue.cc:237] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:12.845106 32735 catalog_manager.cc:5697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 1 to 2, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.120391   445 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:13.121670   660 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:13.130426   371 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:13.132097   669 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:13.261372   445 raft_consensus.cc:1275] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:13.262640   669 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:13.271716   371 raft_consensus.cc:1275] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:13.273571   634 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:13.616822 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:13.632581   521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "16fce92c077847119c7e789058c25852"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40360
I20260709 13:12:13.633085   521 raft_consensus.cc:606] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:13.640491   521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5"
dest_uuid: "16fce92c077847119c7e789058c25852"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40360
I20260709 13:12:13.640967   521 raft_consensus.cc:606] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:13.645675   445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40468
I20260709 13:12:13.646317   445 raft_consensus.cc:606] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:13.651149   445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40468
I20260709 13:12:13.651676   445 raft_consensus.cc:606] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:13.656682   445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
mode: GRACEFUL
new_leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:40468
I20260709 13:12:13.657189   445 raft_consensus.cc:606] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Received request to transfer leadership to TS 937bbecd2c884fbab2e6b549bdc7be83
I20260709 13:12:13.721711   660 raft_consensus.cc:993] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.722155   634 raft_consensus.cc:1081] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.723945   369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38772
I20260709 13:12:13.724607   369 raft_consensus.cc:493] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:13.724975   369 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.729667   369 raft_consensus.cc:515] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.731356   369 leader_election.cc:290] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:13.732148   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:13.732439   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cc17cd9ee3b4c95be367e78c1911fa5" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:13.732712   445 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.732956   521 raft_consensus.cc:3055] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:13.733245   521 raft_consensus.cc:740] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:13.733834   521 consensus_queue.cc:260] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.734616   521 raft_consensus.cc:3060] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.737357   445 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:13.738440   337 leader_election.cc:304] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:13.738845   521 raft_consensus.cc:2468] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:13.739272   590 raft_consensus.cc:2804] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:13.740495   590 raft_consensus.cc:697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:13.741273   590 consensus_queue.cc:237] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.747494 32736 catalog_manager.cc:5697] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 2 to 3, leader changed from 16fce92c077847119c7e789058c25852 (127.31.120.3) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.829312   679 raft_consensus.cc:993] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.829888   619 raft_consensus.cc:1081] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.831423   369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38770
I20260709 13:12:13.831944   369 raft_consensus.cc:493] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:13.832260   369 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.837036   369 raft_consensus.cc:515] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.838703   369 leader_election.cc:290] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:13.839740   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:13.839967   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:13.840306   445 raft_consensus.cc:3055] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:13.840428   521 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.840641   445 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:13.841384   445 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.843016   445 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.847180   521 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:13.848392   337 leader_election.cc:304] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:13.849086   445 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:13.849334   590 raft_consensus.cc:2804] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:13.849902   590 raft_consensus.cc:697] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:13.850673   590 consensus_queue.cc:237] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.855556   619 raft_consensus.cc:993] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.856092   680 raft_consensus.cc:1081] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:13.857880   369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38770
I20260709 13:12:13.858456   369 raft_consensus.cc:493] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:13.858853   369 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:13.858834 32736 catalog_manager.cc:5697] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 2 to 3, leader changed from 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.864099   369 raft_consensus.cc:515] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.866622   369 leader_election.cc:290] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:13.867280   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:13.867630   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:13.867856   445 raft_consensus.cc:3055] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:13.868137   521 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:13.868160   445 raft_consensus.cc:740] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:13.869019   445 consensus_queue.cc:260] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.870090   445 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:13.874574   521 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:13.875697   337 leader_election.cc:304] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:13.876343   445 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 2.
I20260709 13:12:13.876524   590 raft_consensus.cc:2804] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:13.878695   590 raft_consensus.cc:697] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:13.879462   590 consensus_queue.cc:237] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:13.886713 32736 catalog_manager.cc:5697] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 1 to 2, leader changed from 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.100123   679 raft_consensus.cc:993] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:14.100608   679 raft_consensus.cc:1081] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:14.102027   369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38770
I20260709 13:12:14.102563   369 raft_consensus.cc:493] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.102849   369 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:14.107529   369 raft_consensus.cc:515] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.109297   369 leader_election.cc:290] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:14.110121   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:14.110553   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9eda5cd7fa34923b61f966013ceee21" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:14.110669   445 raft_consensus.cc:3055] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:14.111106   521 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:14.111106   445 raft_consensus.cc:740] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:14.112048   445 consensus_queue.cc:260] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.113127   445 raft_consensus.cc:3060] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:14.117527   521 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:14.118830   445 raft_consensus.cc:2468] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 3.
I20260709 13:12:14.118763   337 leader_election.cc:304] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:14.119789   590 raft_consensus.cc:2804] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:14.120177   590 raft_consensus.cc:697] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:14.120810   590 consensus_queue.cc:237] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.127008 32736 catalog_manager.cc:5697] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 2 to 3, leader changed from 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.168148   521 raft_consensus.cc:1275] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.169574   590 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:14.172000   660 raft_consensus.cc:993] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: : Instructing follower 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:14.172551   634 raft_consensus.cc:1081] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Signalling peer 937bbecd2c884fbab2e6b549bdc7be83 to start an election
I20260709 13:12:14.174078   369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e289930f80f44defbb521f29ec049c92"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
 from {username='slave'} at 127.0.0.1:38772
I20260709 13:12:14.174613   369 raft_consensus.cc:493] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.174985   369 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:14.185818   369 raft_consensus.cc:515] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.187057   445 raft_consensus.cc:1275] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.188642   445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:14.189179   445 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:14.189235   687 consensus_queue.cc:1048] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:14.192821   521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e289930f80f44defbb521f29ec049c92" candidate_uuid: "937bbecd2c884fbab2e6b549bdc7be83" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:14.193333   521 raft_consensus.cc:3055] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:14.193584   521 raft_consensus.cc:740] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:14.194300   521 consensus_queue.cc:260] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.195233   521 raft_consensus.cc:3060] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:14.195585   445 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 4.
I20260709 13:12:14.199749   369 leader_election.cc:290] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 4 election: Requested vote from peers 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:14.199708   337 leader_election.cc:304] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d1d0e13ed034afca1c40284daf74edb, 937bbecd2c884fbab2e6b549bdc7be83; no voters: 
I20260709 13:12:14.201040   682 raft_consensus.cc:2804] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:14.201372   521 raft_consensus.cc:2468] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937bbecd2c884fbab2e6b549bdc7be83 in term 4.
I20260709 13:12:14.202426   682 raft_consensus.cc:697] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 LEADER]: Becoming Leader. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:14.203239   682 consensus_queue.cc:237] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:14.209852 32736 catalog_manager.cc:5697] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 reported cstate change: term changed from 3 to 4, leader changed from 16fce92c077847119c7e789058c25852 (127.31.120.3) to 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1). New cstate: current_term: 4 leader_uuid: "937bbecd2c884fbab2e6b549bdc7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.263124   446 raft_consensus.cc:1275] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.263384   521 raft_consensus.cc:1275] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.264726   687 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.265376   682 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.301836   445 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.302983   682 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:14.311779   521 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.313311   687 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:14.600950   521 raft_consensus.cc:1275] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.601219   445 raft_consensus.cc:1275] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:14.602332   590 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:14.602965   675 consensus_queue.cc:1048] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:14.686728   521 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:14.688360   682 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:14.700568   445 raft_consensus.cc:1275] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Refusing update from remote peer 937bbecd2c884fbab2e6b549bdc7be83: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:14.701977   590 consensus_queue.cc:1048] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:16.665098 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:16.673612   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:16.674551   369 raft_consensus.cc:606] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Received request to transfer leadership to TS 16fce92c077847119c7e789058c25852
I20260709 13:12:16.677263   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c407db8719844536a983858e0d68d43b"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:16.677819   369 raft_consensus.cc:606] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Received request to transfer leadership to TS 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:16.680120   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:16.680599   369 raft_consensus.cc:606] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Received request to transfer leadership to TS 5d1d0e13ed034afca1c40284daf74edb
I20260709 13:12:16.682725   369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf"
dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
mode: GRACEFUL
new_leader_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:38784
I20260709 13:12:16.683175   369 raft_consensus.cc:606] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Received request to transfer leadership to TS 16fce92c077847119c7e789058c25852
I20260709 13:12:16.684166 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:12:16.684633 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:16.730165   687 raft_consensus.cc:993] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:16.730715   699 raft_consensus.cc:1081] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Signalling peer 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:16.732400   446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c407db8719844536a983858e0d68d43b"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:40450
I20260709 13:12:16.733091   446 raft_consensus.cc:493] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:16.733559   446 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:16.739280   446 raft_consensus.cc:515] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:16.741209   446 leader_election.cc:290] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Requested vote from peers 16fce92c077847119c7e789058c25852 (127.31.120.3:45563), 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979)
I20260709 13:12:16.742429   369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:16.742517   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c407db8719844536a983858e0d68d43b" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:16.743472   369 raft_consensus.cc:3055] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:16.743472   520 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:16.744021   369 raft_consensus.cc:740] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:16.745009   369 consensus_queue.cc:260] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:16.746490   369 raft_consensus.cc:3060] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:16.751991   520 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:16.753564   412 leader_election.cc:304] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:16.754436   369 raft_consensus.cc:2468] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 2.
I20260709 13:12:16.754521   679 raft_consensus.cc:2804] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:16.756021   679 raft_consensus.cc:697] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:16.756942   679 consensus_queue.cc:237] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } }
I20260709 13:12:16.764959 32737 catalog_manager.cc:5697] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 1 to 2, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 2 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:16.834589   687 raft_consensus.cc:993] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:16.835093   699 raft_consensus.cc:1081] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Signalling peer 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:16.836872   520 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06834392e2b44bd58010de9e2debda51"
dest_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:40346
I20260709 13:12:16.837366   520 raft_consensus.cc:493] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:16.837703   520 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.842821   520 raft_consensus.cc:515] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.844638   520 leader_election.cc:290] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 4 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:16.845618   369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:16.845721   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06834392e2b44bd58010de9e2debda51" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:16.846150   369 raft_consensus.cc:3055] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:16.846387   446 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.846553   369 raft_consensus.cc:740] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:16.847333   369 consensus_queue.cc:260] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.848383   369 raft_consensus.cc:3060] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.852285   446 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 4.
I20260709 13:12:16.853835   369 raft_consensus.cc:2468] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 4.
I20260709 13:12:16.853837   487 leader_election.cc:304] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:16.855123   634 raft_consensus.cc:2804] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:16.856386   634 raft_consensus.cc:697] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 4 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:16.857216   634 consensus_queue.cc:237] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.864876 32737 catalog_manager.cc:5697] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 3 to 4, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:16.919772   687 raft_consensus.cc:993] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:16.920217   699 raft_consensus.cc:1081] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Signalling peer 16fce92c077847119c7e789058c25852 to start an election
I20260709 13:12:16.921684   520 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf"
dest_uuid: "16fce92c077847119c7e789058c25852"
 from {username='slave'} at 127.0.0.1:40346
I20260709 13:12:16.922238   520 raft_consensus.cc:493] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:16.922508   520 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:16.927062   520 raft_consensus.cc:515] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.928613   520 leader_election.cc:290] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 3 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2:32919)
I20260709 13:12:16.929641   369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:16.929806   446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e0bd8e52594d5f85bde9c940da47bf" candidate_uuid: "16fce92c077847119c7e789058c25852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
I20260709 13:12:16.930289   369 raft_consensus.cc:3055] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:16.930342   446 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:16.930650   369 raft_consensus.cc:740] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:16.931568   369 consensus_queue.cc:260] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.932813   369 raft_consensus.cc:3060] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:16.936533   446 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 3.
I20260709 13:12:16.937603   487 leader_election.cc:304] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:16.938489   634 raft_consensus.cc:2804] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:16.938875   634 raft_consensus.cc:697] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Becoming Leader. State: Replica: 16fce92c077847119c7e789058c25852, State: Running, Role: LEADER
I20260709 13:12:16.939241   369 raft_consensus.cc:2468] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16fce92c077847119c7e789058c25852 in term 3.
I20260709 13:12:16.939735   634 consensus_queue.cc:237] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:16.946352 32737 catalog_manager.cc:5697] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 reported cstate change: term changed from 2 to 3, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 16fce92c077847119c7e789058c25852 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "16fce92c077847119c7e789058c25852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.146490   687 raft_consensus.cc:993] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: : Instructing follower 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:17.146921   699 raft_consensus.cc:1081] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Signalling peer 5d1d0e13ed034afca1c40284daf74edb to start an election
I20260709 13:12:17.148458   445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51"
dest_uuid: "5d1d0e13ed034afca1c40284daf74edb"
 from {username='slave'} at 127.0.0.1:40450
I20260709 13:12:17.148972   445 raft_consensus.cc:493] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:17.149277   445 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:17.153977   445 raft_consensus.cc:515] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:17.155450   445 leader_election.cc:290] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 3 election: Requested vote from peers 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979), 16fce92c077847119c7e789058c25852 (127.31.120.3:45563)
I20260709 13:12:17.156353   369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "937bbecd2c884fbab2e6b549bdc7be83"
I20260709 13:12:17.156589   520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed5f57df654aeda85acfb967bfed51" candidate_uuid: "5d1d0e13ed034afca1c40284daf74edb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16fce92c077847119c7e789058c25852"
I20260709 13:12:17.156961   369 raft_consensus.cc:3055] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:17.157050   520 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:17.157351   369 raft_consensus.cc:740] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 937bbecd2c884fbab2e6b549bdc7be83, State: Running, Role: LEADER
I20260709 13:12:17.158202   369 consensus_queue.cc:260] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:17.159404   369 raft_consensus.cc:3060] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:17.162305   520 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 3.
I20260709 13:12:17.163389   412 leader_election.cc:304] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16fce92c077847119c7e789058c25852, 5d1d0e13ed034afca1c40284daf74edb; no voters: 
I20260709 13:12:17.164162   679 raft_consensus.cc:2804] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:17.164572   679 raft_consensus.cc:697] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 LEADER]: Becoming Leader. State: Replica: 5d1d0e13ed034afca1c40284daf74edb, State: Running, Role: LEADER
I20260709 13:12:17.164857   369 raft_consensus.cc:2468] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d1d0e13ed034afca1c40284daf74edb in term 3.
I20260709 13:12:17.165521   679 consensus_queue.cc:237] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } }
I20260709 13:12:17.171569   520 raft_consensus.cc:1275] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.172775   679 consensus_queue.cc:1048] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:17.172701 32737 catalog_manager.cc:5697] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb reported cstate change: term changed from 2 to 3, leader changed from 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1) to 5d1d0e13ed034afca1c40284daf74edb (127.31.120.2). New cstate: current_term: 3 leader_uuid: "5d1d0e13ed034afca1c40284daf74edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:17.189090   369 raft_consensus.cc:1275] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.191458   712 consensus_queue.cc:1048] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:17.256776   445 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:17.258333   634 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:17.268508   369 raft_consensus.cc:1275] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:17.269829   634 consensus_queue.cc:1048] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:17.552729   445 raft_consensus.cc:1275] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:17.554004   634 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d1d0e13ed034afca1c40284daf74edb" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:17.564580   369 raft_consensus.cc:1275] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Refusing update from remote peer 16fce92c077847119c7e789058c25852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:17.565311   371 raft_consensus.cc:1275] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:17.567046   712 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:17.567164   634 consensus_queue.cc:1048] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "937bbecd2c884fbab2e6b549bdc7be83" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:17.579420   520 raft_consensus.cc:1275] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Refusing update from remote peer 5d1d0e13ed034afca1c40284daf74edb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:17.580817   713 consensus_queue.cc:1048] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "16fce92c077847119c7e789058c25852" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:17.685484 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:17.691324 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:12:17.717111 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:17.717903 32224 tablet_replica.cc:333] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.718650 32224 raft_consensus.cc:2243] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.719144 32224 raft_consensus.cc:2272] T 55059494f790453b85cdf6ee41344040 P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.721084 32224 tablet_replica.cc:333] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.721632 32224 raft_consensus.cc:2243] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.722046 32224 raft_consensus.cc:2272] T c407db8719844536a983858e0d68d43b P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.724315 32224 tablet_replica.cc:333] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.724977 32224 raft_consensus.cc:2243] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.725704 32224 raft_consensus.cc:2272] T e289930f80f44defbb521f29ec049c92 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.727380 32224 tablet_replica.cc:333] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.727912 32224 raft_consensus.cc:2243] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.728565 32224 raft_consensus.cc:2272] T 72ee08ff22b14a249371c0a58f076903 P 937bbecd2c884fbab2e6b549bdc7be83 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.730224 32224 tablet_replica.cc:333] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.730702 32224 raft_consensus.cc:2243] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.731089 32224 raft_consensus.cc:2272] T f3ed5f57df654aeda85acfb967bfed51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.733799 32224 tablet_replica.cc:333] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.734309 32224 raft_consensus.cc:2243] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.734695 32224 raft_consensus.cc:2272] T 06834392e2b44bd58010de9e2debda51 P 937bbecd2c884fbab2e6b549bdc7be83 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.736788 32224 tablet_replica.cc:333] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.737275 32224 raft_consensus.cc:2243] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.737649 32224 raft_consensus.cc:2272] T e2497ebb53c7472292e77dc6e8c8bb6f P 937bbecd2c884fbab2e6b549bdc7be83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.739212 32224 tablet_replica.cc:333] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.739758 32224 raft_consensus.cc:2243] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.740554 32224 raft_consensus.cc:2272] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.742316 32224 tablet_replica.cc:333] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.742792 32224 raft_consensus.cc:2243] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.743741 32224 raft_consensus.cc:2272] T c9eda5cd7fa34923b61f966013ceee21 P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.745484 32224 tablet_replica.cc:333] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83: stopping tablet replica
I20260709 13:12:17.745935 32224 raft_consensus.cc:2243] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.746320 32224 raft_consensus.cc:2272] T f4e0bd8e52594d5f85bde9c940da47bf P 937bbecd2c884fbab2e6b549bdc7be83 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.776113 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
I20260709 13:12:17.796056 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:12:17.806385   410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40979: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260709 13:12:17.812943   410 consensus_peers.cc:597] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb -> Peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Couldn't send request to peer 937bbecd2c884fbab2e6b549bdc7be83. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:17.828553 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:17.829315 32224 tablet_replica.cc:333] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.830353 32224 raft_consensus.cc:2243] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.830976 32224 raft_consensus.cc:2272] T e289930f80f44defbb521f29ec049c92 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.832878 32224 tablet_replica.cc:333] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.833364 32224 raft_consensus.cc:2243] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.834072 32224 raft_consensus.cc:2272] T e2497ebb53c7472292e77dc6e8c8bb6f P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.835973 32224 tablet_replica.cc:333] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.836426 32224 raft_consensus.cc:2243] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.836771 32224 raft_consensus.cc:2272] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.838322 32224 tablet_replica.cc:333] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.838814 32224 raft_consensus.cc:2243] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.839149 32224 raft_consensus.cc:2272] T 55059494f790453b85cdf6ee41344040 P 5d1d0e13ed034afca1c40284daf74edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.841020 32224 tablet_replica.cc:333] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.841470 32224 raft_consensus.cc:2243] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.841825 32224 raft_consensus.cc:2272] T f4e0bd8e52594d5f85bde9c940da47bf P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.844099 32224 tablet_replica.cc:333] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.844568 32224 raft_consensus.cc:2243] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.845129 32224 raft_consensus.cc:2272] T c9eda5cd7fa34923b61f966013ceee21 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.847308 32224 tablet_replica.cc:333] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.848024 32224 raft_consensus.cc:2243] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.848795 32224 raft_consensus.cc:2272] T f3ed5f57df654aeda85acfb967bfed51 P 5d1d0e13ed034afca1c40284daf74edb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.850596 32224 tablet_replica.cc:333] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.851068 32224 raft_consensus.cc:2243] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.851428 32224 raft_consensus.cc:2272] T 72ee08ff22b14a249371c0a58f076903 P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.853257 32224 tablet_replica.cc:333] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.853667 32224 raft_consensus.cc:2243] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.854156 32224 raft_consensus.cc:2272] T 06834392e2b44bd58010de9e2debda51 P 5d1d0e13ed034afca1c40284daf74edb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.856441 32224 tablet_replica.cc:333] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb: stopping tablet replica
I20260709 13:12:17.856855 32224 raft_consensus.cc:2243] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.857501 32224 raft_consensus.cc:2272] T c407db8719844536a983858e0d68d43b P 5d1d0e13ed034afca1c40284daf74edb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:17.881699   485 consensus_peers.cc:597] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 -> Peer 937bbecd2c884fbab2e6b549bdc7be83 (127.31.120.1:40979): Couldn't send request to peer 937bbecd2c884fbab2e6b549bdc7be83. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:17.885577 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
I20260709 13:12:17.903241 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:12:17.929400 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:17.930275 32224 tablet_replica.cc:333] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.930805 32224 raft_consensus.cc:2243] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.931258 32224 raft_consensus.cc:2272] T e289930f80f44defbb521f29ec049c92 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.932960 32224 tablet_replica.cc:333] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.933495 32224 raft_consensus.cc:2243] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.934255 32224 raft_consensus.cc:2272] T 55059494f790453b85cdf6ee41344040 P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.935992 32224 tablet_replica.cc:333] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.936488 32224 raft_consensus.cc:2243] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.936874 32224 raft_consensus.cc:2272] T c407db8719844536a983858e0d68d43b P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.938778 32224 tablet_replica.cc:333] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.939332 32224 raft_consensus.cc:2243] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.940125 32224 raft_consensus.cc:2272] T 06834392e2b44bd58010de9e2debda51 P 16fce92c077847119c7e789058c25852 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.942723 32224 tablet_replica.cc:333] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.943248 32224 raft_consensus.cc:2243] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.943859 32224 raft_consensus.cc:2272] T f3ed5f57df654aeda85acfb967bfed51 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.945971 32224 tablet_replica.cc:333] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.946472 32224 raft_consensus.cc:2243] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.946914 32224 raft_consensus.cc:2272] T 72ee08ff22b14a249371c0a58f076903 P 16fce92c077847119c7e789058c25852 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.948591 32224 tablet_replica.cc:333] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.949090 32224 raft_consensus.cc:2243] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.949489 32224 raft_consensus.cc:2272] T e2497ebb53c7472292e77dc6e8c8bb6f P 16fce92c077847119c7e789058c25852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.951087 32224 tablet_replica.cc:333] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.951562 32224 raft_consensus.cc:2243] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.952037 32224 raft_consensus.cc:2272] T 2cc17cd9ee3b4c95be367e78c1911fa5 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.953622 32224 tablet_replica.cc:333] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.954071 32224 raft_consensus.cc:2243] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:17.954778 32224 raft_consensus.cc:2272] T f4e0bd8e52594d5f85bde9c940da47bf P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.956821 32224 tablet_replica.cc:333] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852: stopping tablet replica
I20260709 13:12:17.957289 32224 raft_consensus.cc:2243] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:17.957763 32224 raft_consensus.cc:2272] T c9eda5cd7fa34923b61f966013ceee21 P 16fce92c077847119c7e789058c25852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:17.978803 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:12:17.996560 32224 master.cc:562] Master@127.31.120.62:44129 shutting down...
I20260709 13:12:18.016536 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:18.017068 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.017396 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P b7131067716b49059f45890809976e34: stopping tablet replica
I20260709 13:12:18.037474 32224 master.cc:584] Master@127.31.120.62:44129 shutdown complete.
16fce92c077847119c7e789058c25852  1
5d1d0e13ed034afca1c40284daf74edb  4
937bbecd2c884fbab2e6b549bdc7be83  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10353 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:12:18.080587 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:45937
I20260709 13:12:18.081691 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:18.088059   729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.088228   731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.088050   728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.090361 32224 server_base.cc:1061] running on GCE node
I20260709 13:12:18.091166 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:18.091342 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:18.091454 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602738091442 us; error 0 us; skew 500 ppm
I20260709 13:12:18.092129 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:18.094700 32224 webserver.cc:533] Webserver started at http://127.31.120.62:46315/ using document root <none> and password file <none>
I20260709 13:12:18.095126 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:18.095278 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:18.095506 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:18.096629 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "ad0b4d62f81540a8878fe1d1c1552895"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.101266 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:18.104821   736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.105629 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:12:18.105930 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "ad0b4d62f81540a8878fe1d1c1552895"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.106280 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:18.123793 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:18.125061 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:18.164392 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:45937
I20260709 13:12:18.164510   787 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:45937 every 8 connection(s)
I20260709 13:12:18.168371   788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.178169   788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Bootstrap starting.
I20260709 13:12:18.182751   788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.186957   788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: No bootstrap required, opened a new log
I20260709 13:12:18.189082   788 raft_consensus.cc:359] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER }
I20260709 13:12:18.189482   788 raft_consensus.cc:385] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.189742   788 raft_consensus.cc:740] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0b4d62f81540a8878fe1d1c1552895, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.190408   788 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER }
I20260709 13:12:18.190920   788 raft_consensus.cc:399] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:18.191124   788 raft_consensus.cc:493] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:18.191417   788 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.195783   788 raft_consensus.cc:515] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER }
I20260709 13:12:18.196285   788 leader_election.cc:304] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad0b4d62f81540a8878fe1d1c1552895; no voters: 
I20260709 13:12:18.197407   788 leader_election.cc:290] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:18.197794   791 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.199239   791 raft_consensus.cc:697] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 1 LEADER]: Becoming Leader. State: Replica: ad0b4d62f81540a8878fe1d1c1552895, State: Running, Role: LEADER
I20260709 13:12:18.199940   791 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER }
I20260709 13:12:18.200476   788 sys_catalog.cc:565] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:18.205142   793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad0b4d62f81540a8878fe1d1c1552895. Latest consensus state: current_term: 1 leader_uuid: "ad0b4d62f81540a8878fe1d1c1552895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER } }
I20260709 13:12:18.205361   792 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad0b4d62f81540a8878fe1d1c1552895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0b4d62f81540a8878fe1d1c1552895" member_type: VOTER } }
I20260709 13:12:18.205852   793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:18.206123   792 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:18.208913   799 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:18.214233   799 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:18.217006 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:18.223202   799 catalog_manager.cc:1383] Generated new cluster ID: ffd6cd02e47d47d588a3c5495fd0936f
I20260709 13:12:18.223498   799 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:18.243072   799 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:18.244338   799 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:18.257628   799 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Generated new TSK 0
I20260709 13:12:18.258237   799 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:18.283430 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:18.289924   809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.290920   810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.295415 32224 server_base.cc:1061] running on GCE node
W20260709 13:12:18.295058   812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.299387 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:18.299669 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:18.299917 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602738299870 us; error 0 us; skew 500 ppm
I20260709 13:12:18.300499 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:18.302929 32224 webserver.cc:533] Webserver started at http://127.31.120.1:45397/ using document root <none> and password file <none>
I20260709 13:12:18.303426 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:18.303654 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:18.303941 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:18.305047 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "64f2cb5b0e174002811ef197f9eec829"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.309706 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:18.313601   817 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.314303 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:18.314615 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "64f2cb5b0e174002811ef197f9eec829"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.314944 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:18.347748 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:18.349004 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:18.350533 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:18.352905 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:18.353124 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.353384 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:18.353570 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.396742 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:43377
I20260709 13:12:18.396849   879 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:43377 every 8 connection(s)
I20260709 13:12:18.401753 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:18.409802   884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.411059   885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.413694 32224 server_base.cc:1061] running on GCE node
W20260709 13:12:18.413955   887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.415413 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:18.415664 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:18.415851 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602738415838 us; error 0 us; skew 500 ppm
I20260709 13:12:18.416460 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:18.418741   880 heartbeater.cc:344] Connected to a master server at 127.31.120.62:45937
I20260709 13:12:18.419154   880 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:18.419425 32224 webserver.cc:533] Webserver started at http://127.31.120.2:44485/ using document root <none> and password file <none>
I20260709 13:12:18.419989 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:18.420176 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:18.420073   880 heartbeater.cc:507] Master 127.31.120.62:45937 requested a full tablet report, sending...
I20260709 13:12:18.420584 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:18.421880 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "91ff541db50a49efa03669dfd777fc8e"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.422546   753 ts_manager.cc:194] Registered new tserver with Master: 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:18.424503   753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55304
I20260709 13:12:18.427549 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:12:18.431315   892 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.432401 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:12:18.432746 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "91ff541db50a49efa03669dfd777fc8e"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.433049 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:18.448087 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:18.449383 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:18.451004 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:18.453488 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:18.453758 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.454033 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:18.454273 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.497771 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:34547
I20260709 13:12:18.497862   954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:34547 every 8 connection(s)
I20260709 13:12:18.503105 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:18.512030   959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.513746   958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:18.515362   961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:18.516289 32224 server_base.cc:1061] running on GCE node
I20260709 13:12:18.517346 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:18.517596 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:18.517804 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602738517788 us; error 0 us; skew 500 ppm
I20260709 13:12:18.518190   955 heartbeater.cc:344] Connected to a master server at 127.31.120.62:45937
I20260709 13:12:18.518558 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:18.518608   955 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:18.519778   955 heartbeater.cc:507] Master 127.31.120.62:45937 requested a full tablet report, sending...
I20260709 13:12:18.521795 32224 webserver.cc:533] Webserver started at http://127.31.120.3:35549/ using document root <none> and password file <none>
I20260709 13:12:18.522106   753 ts_manager.cc:194] Registered new tserver with Master: 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:18.522516 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:18.522779 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:18.523063 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:18.523530   753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55314
I20260709 13:12:18.524401 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "3bb5726887244caca146e176cc6bd070"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.529611 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:12:18.533066   966 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.533793 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:18.534098 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "3bb5726887244caca146e176cc6bd070"
format_stamp: "Formatted at 2026-07-09 13:12:18 on dist-test-slave-pm98"
I20260709 13:12:18.534474 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:18.550096 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:18.551525 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:18.553293 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:18.555802 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:18.556049 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.556295 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:18.556483 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:18.596624 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:44411
I20260709 13:12:18.596761  1028 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:44411 every 8 connection(s)
I20260709 13:12:18.610796  1029 heartbeater.cc:344] Connected to a master server at 127.31.120.62:45937
I20260709 13:12:18.611220  1029 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:18.612067  1029 heartbeater.cc:507] Master 127.31.120.62:45937 requested a full tablet report, sending...
I20260709 13:12:18.613978   753 ts_manager.cc:194] Registered new tserver with Master: 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:18.614560 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014636727s
I20260709 13:12:18.615376   753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55328
I20260709 13:12:18.641346   753 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55332:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:18.647728   753 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:18.854398   920 tablet_service.cc:1511] Processing CreateTablet for tablet bedf1cefa52d44d884e14c5261c6437a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:18.855997   920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedf1cefa52d44d884e14c5261c6437a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.856230   993 tablet_service.cc:1511] Processing CreateTablet for tablet 10b0600fc3f340eb999b12ed484fcdcb (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:18.857165   919 tablet_service.cc:1511] Processing CreateTablet for tablet 10b0600fc3f340eb999b12ed484fcdcb (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:18.857777   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b0600fc3f340eb999b12ed484fcdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.858585   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b0600fc3f340eb999b12ed484fcdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.859089   845 tablet_service.cc:1511] Processing CreateTablet for tablet bedf1cefa52d44d884e14c5261c6437a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:18.858541   842 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf198f3795b43ff951a11167bc21580 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:18.860697   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedf1cefa52d44d884e14c5261c6437a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.860877   918 tablet_service.cc:1511] Processing CreateTablet for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:18.861563   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf198f3795b43ff951a11167bc21580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.862231   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.862907   991 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf198f3795b43ff951a11167bc21580 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:18.864272   991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf198f3795b43ff951a11167bc21580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.864780   994 tablet_service.cc:1511] Processing CreateTablet for tablet bedf1cefa52d44d884e14c5261c6437a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:18.865989   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedf1cefa52d44d884e14c5261c6437a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.868825   917 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf198f3795b43ff951a11167bc21580 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:18.870350   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf198f3795b43ff951a11167bc21580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.870667   843 tablet_service.cc:1511] Processing CreateTablet for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:18.860702   841 tablet_service.cc:1511] Processing CreateTablet for tablet bdca38d5137a4409b2e61eaf2e13fd7b (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:18.868871   844 tablet_service.cc:1511] Processing CreateTablet for tablet 10b0600fc3f340eb999b12ed484fcdcb (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:18.873085   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.872696   916 tablet_service.cc:1511] Processing CreateTablet for tablet bdca38d5137a4409b2e61eaf2e13fd7b (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:18.857713   992 tablet_service.cc:1511] Processing CreateTablet for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:18.874922   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdca38d5137a4409b2e61eaf2e13fd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.875527   992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.876571   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b0600fc3f340eb999b12ed484fcdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.879045   840 tablet_service.cc:1511] Processing CreateTablet for tablet 6cfaab49f56546a49978c6055f57640f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:18.880434   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfaab49f56546a49978c6055f57640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.880209   915 tablet_service.cc:1511] Processing CreateTablet for tablet 6cfaab49f56546a49978c6055f57640f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:18.881553   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfaab49f56546a49978c6055f57640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.885743   839 tablet_service.cc:1511] Processing CreateTablet for tablet 7de58b0d5ad94d869fac3b0491143543 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:18.887141   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de58b0d5ad94d869fac3b0491143543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.887738   914 tablet_service.cc:1511] Processing CreateTablet for tablet 7de58b0d5ad94d869fac3b0491143543 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:18.889134   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de58b0d5ad94d869fac3b0491143543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.889793   838 tablet_service.cc:1511] Processing CreateTablet for tablet 01fc410be7f74515adce264a85731b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:18.891109   838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01fc410be7f74515adce264a85731b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.893450   990 tablet_service.cc:1511] Processing CreateTablet for tablet bdca38d5137a4409b2e61eaf2e13fd7b (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:18.894279   837 tablet_service.cc:1511] Processing CreateTablet for tablet c6e1447d2c0c4822aac226064c087c42 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:18.894883   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdca38d5137a4409b2e61eaf2e13fd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.896162   988 tablet_service.cc:1511] Processing CreateTablet for tablet 7de58b0d5ad94d869fac3b0491143543 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:18.896801   989 tablet_service.cc:1511] Processing CreateTablet for tablet 6cfaab49f56546a49978c6055f57640f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:18.897573   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de58b0d5ad94d869fac3b0491143543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.901296   985 tablet_service.cc:1511] Processing CreateTablet for tablet 35b46c75a1e049dd910275709bd62839 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:18.902724   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b46c75a1e049dd910275709bd62839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.904811   836 tablet_service.cc:1511] Processing CreateTablet for tablet 35b46c75a1e049dd910275709bd62839 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:18.905366   912 tablet_service.cc:1511] Processing CreateTablet for tablet c6e1447d2c0c4822aac226064c087c42 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:18.906256   836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b46c75a1e049dd910275709bd62839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.907799   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e1447d2c0c4822aac226064c087c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.908499   986 tablet_service.cc:1511] Processing CreateTablet for tablet c6e1447d2c0c4822aac226064c087c42 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:18.910609   987 tablet_service.cc:1511] Processing CreateTablet for tablet 01fc410be7f74515adce264a85731b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:18.904811   911 tablet_service.cc:1511] Processing CreateTablet for tablet 35b46c75a1e049dd910275709bd62839 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:18.913769   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b46c75a1e049dd910275709bd62839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.902756   913 tablet_service.cc:1511] Processing CreateTablet for tablet 01fc410be7f74515adce264a85731b90 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:18.919234   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01fc410be7f74515adce264a85731b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.924027   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfaab49f56546a49978c6055f57640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.930063   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01fc410be7f74515adce264a85731b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.935473   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdca38d5137a4409b2e61eaf2e13fd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.955976   837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e1447d2c0c4822aac226064c087c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.963873   986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e1447d2c0c4822aac226064c087c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.996603  1044 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:19.007462   994 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:19.008895   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe55b34a7fd4408b9a1e51ded1fd129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:18.947409   993 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b9f004a6f4293a658cca1bb92e319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:19.015540   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b9f004a6f4293a658cca1bb92e319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.028589   919 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b9f004a6f4293a658cca1bb92e319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:19.068559   915 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:19.070031  1044 tablet_bootstrap.cc:654] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.070165   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe55b34a7fd4408b9a1e51ded1fd129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.064786   989 tablet_service.cc:1511] Processing CreateTablet for tablet 4997f2c7b33048a5b010a4e0c7866fa2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:19.073845   985 tablet_service.cc:1511] Processing CreateTablet for tablet 9794c08fc6b2428b981acd6c3c238b5f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:19.075337   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4997f2c7b33048a5b010a4e0c7866fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.076037   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9794c08fc6b2428b981acd6c3c238b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.081902   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b9f004a6f4293a658cca1bb92e319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.097931   914 tablet_service.cc:1511] Processing CreateTablet for tablet 4997f2c7b33048a5b010a4e0c7866fa2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:19.098537   989 tablet_service.cc:1511] Processing CreateTablet for tablet 85bbb1b886474a538a2f1a32dd91b560 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:19.108848   915 tablet_service.cc:1511] Processing CreateTablet for tablet 9794c08fc6b2428b981acd6c3c238b5f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:19.131778   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bbb1b886474a538a2f1a32dd91b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.141757   842 tablet_service.cc:1511] Processing CreateTablet for tablet 9794c08fc6b2428b981acd6c3c238b5f (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:19.143080   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9794c08fc6b2428b981acd6c3c238b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.116422   991 tablet_service.cc:1511] Processing CreateTablet for tablet 731bda621b364aaa81d868d0cef5e739 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:19.156999  1046 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:19.160909  1045 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:19.147516   837 tablet_service.cc:1511] Processing CreateTablet for tablet 85bbb1b886474a538a2f1a32dd91b560 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:19.162412   837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bbb1b886474a538a2f1a32dd91b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.166338   990 tablet_service.cc:1511] Processing CreateTablet for tablet 6874f81648724ea980572af4b6fa79d2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:19.167641   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874f81648724ea980572af4b6fa79d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.167441   836 tablet_service.cc:1511] Processing CreateTablet for tablet 731bda621b364aaa81d868d0cef5e739 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:19.181265   920 tablet_service.cc:1511] Processing CreateTablet for tablet cdfea09a06ff4100a0b80619f35f7008 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:19.199056  1045 tablet_bootstrap.cc:654] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.373505   913 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9d8e1d04724f1ba621ecf90cc49b82 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:19.374354   993 tablet_service.cc:1511] Processing CreateTablet for tablet d548ef2981894fe6972da35c997f5cff (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:19.178818   918 tablet_service.cc:1511] Processing CreateTablet for tablet 85bbb1b886474a538a2f1a32dd91b560 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:19.184783   911 tablet_service.cc:1511] Processing CreateTablet for tablet 731bda621b364aaa81d868d0cef5e739 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:19.170250   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4997f2c7b33048a5b010a4e0c7866fa2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:19.213668   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:19.387552   920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdfea09a06ff4100a0b80619f35f7008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.169538   840 tablet_service.cc:1511] Processing CreateTablet for tablet cdfea09a06ff4100a0b80619f35f7008 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:19.192132   991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bda621b364aaa81d868d0cef5e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.389906   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d548ef2981894fe6972da35c997f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.218750   994 tablet_service.cc:1511] Processing CreateTablet for tablet cdfea09a06ff4100a0b80619f35f7008 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:19.391439   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdfea09a06ff4100a0b80619f35f7008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.201406  1044 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:19.395546  1044 ts_tablet_manager.cc:1403] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.399s	user 0.026s	sys 0.018s
I20260709 13:12:19.281679   912 tablet_service.cc:1511] Processing CreateTablet for tablet 91555b4ec53349c287c5221b9030c23a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:19.300077   838 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb0c412ac6a42fe84d4b2ce85773e26 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:19.404181  1044 raft_consensus.cc:359] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.404930  1044 raft_consensus.cc:385] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:19.405234  1044 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:19.405953  1044 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.413426   993 tablet_service.cc:1511] Processing CreateTablet for tablet c72e2f13b3404583a1c34e15098c42b9 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:19.414939   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72e2f13b3404583a1c34e15098c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.316475   839 tablet_service.cc:1511] Processing CreateTablet for tablet 91555b4ec53349c287c5221b9030c23a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:19.319157   845 tablet_service.cc:1511] Processing CreateTablet for tablet 4997f2c7b33048a5b010a4e0c7866fa2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:19.416667  1029 heartbeater.cc:499] Master 127.31.120.62:45937 was elected leader, sending a full tablet report...
I20260709 13:12:19.320851   836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bda621b364aaa81d868d0cef5e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.138020   841 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b9f004a6f4293a658cca1bb92e319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:19.417699   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b9f004a6f4293a658cca1bb92e319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.210254   988 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb0c412ac6a42fe84d4b2ce85773e26 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:19.418627   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb0c412ac6a42fe84d4b2ce85773e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.331854   919 tablet_service.cc:1511] Processing CreateTablet for tablet 6874f81648724ea980572af4b6fa79d2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:19.337014   917 tablet_service.cc:1511] Processing CreateTablet for tablet e65c03394bf04e2fb03562b27d85f5b3 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:19.419870   842 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9d8e1d04724f1ba621ecf90cc49b82 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:19.421290  1049 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.421780  1049 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.426198   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9d8e1d04724f1ba621ecf90cc49b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.427331   880 heartbeater.cc:499] Master 127.31.120.62:45937 was elected leader, sending a full tablet report...
I20260709 13:12:19.427953   838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb0c412ac6a42fe84d4b2ce85773e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.428552   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4997f2c7b33048a5b010a4e0c7866fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.431947   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdfea09a06ff4100a0b80619f35f7008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.432914  1044 ts_tablet_manager.cc:1434] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:12:19.440490  1049 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:19.453063   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91555b4ec53349c287c5221b9030c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.433902  1044 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:19.346428   990 tablet_service.cc:1511] Processing CreateTablet for tablet 91555b4ec53349c287c5221b9030c23a (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:19.347337   843 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:19.350643   992 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9d8e1d04724f1ba621ecf90cc49b82 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:19.351850   837 tablet_service.cc:1511] Processing CreateTablet for tablet e65c03394bf04e2fb03562b27d85f5b3 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:19.356658   987 tablet_service.cc:1511] Processing CreateTablet for tablet ef39166ba76a41d18b3770b743ea3488 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:12:19.361505   916 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb0c412ac6a42fe84d4b2ce85773e26 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:19.363698   989 tablet_service.cc:1511] Processing CreateTablet for tablet 9155790402004e4f930983841dfd8c64 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:19.370962   844 tablet_service.cc:1511] Processing CreateTablet for tablet 6874f81648724ea980572af4b6fa79d2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:19.376223   985 tablet_service.cc:1511] Processing CreateTablet for tablet 3b871863269841f7b0aa8baf5e640865 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:19.379660  1045 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:19.473785  1045 ts_tablet_manager.cc:1403] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.334s	user 0.012s	sys 0.000s
I20260709 13:12:19.474977   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef39166ba76a41d18b3770b743ea3488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.477219   992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9d8e1d04724f1ba621ecf90cc49b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.173454   986 tablet_service.cc:1511] Processing CreateTablet for tablet e65c03394bf04e2fb03562b27d85f5b3 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:19.479189   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe55b34a7fd4408b9a1e51ded1fd129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.476758  1045 raft_consensus.cc:359] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.388284   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9794c08fc6b2428b981acd6c3c238b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.480541   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65c03394bf04e2fb03562b27d85f5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.480844  1045 raft_consensus.cc:385] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:19.488611  1045 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:19.488833   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb0c412ac6a42fe84d4b2ce85773e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.488060   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874f81648724ea980572af4b6fa79d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.492019   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91555b4ec53349c287c5221b9030c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.492681   837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65c03394bf04e2fb03562b27d85f5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.492717   986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65c03394bf04e2fb03562b27d85f5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.496068   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bda621b364aaa81d868d0cef5e739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.496286   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b871863269841f7b0aa8baf5e640865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.497730   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155790402004e4f930983841dfd8c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.498421  1044 tablet_bootstrap.cc:654] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.492374  1045 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.501368   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6874f81648724ea980572af4b6fa79d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.509369   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91555b4ec53349c287c5221b9030c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.523845   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9d8e1d04724f1ba621ecf90cc49b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.526430   955 heartbeater.cc:499] Master 127.31.120.62:45937 was elected leader, sending a full tablet report...
I20260709 13:12:19.526628   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bbb1b886474a538a2f1a32dd91b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.543404  1045 ts_tablet_manager.cc:1434] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.005s
I20260709 13:12:19.544759  1045 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:19.546432  1046 tablet_bootstrap.cc:654] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.565888   988 tablet_service.cc:1511] Processing CreateTablet for tablet d3d5a097d0fe49bab1a3ec1f78f4164d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:19.567179   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d5a097d0fe49bab1a3ec1f78f4164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.568413   845 tablet_service.cc:1511] Processing CreateTablet for tablet ef39166ba76a41d18b3770b743ea3488 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:12:19.569604   838 tablet_service.cc:1511] Processing CreateTablet for tablet 9155790402004e4f930983841dfd8c64 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:19.584177   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:19.585924   969 leader_election.cc:343] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:19.569648   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef39166ba76a41d18b3770b743ea3488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.590883   991 tablet_service.cc:1511] Processing CreateTablet for tablet b8c347b6586f486192f48c46974a2526 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:19.591390   838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155790402004e4f930983841dfd8c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.592262   991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c347b6586f486192f48c46974a2526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.602392   914 tablet_service.cc:1511] Processing CreateTablet for tablet ef39166ba76a41d18b3770b743ea3488 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:12:19.603742   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef39166ba76a41d18b3770b743ea3488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.605427   993 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee81d93e84c4c689b02deb555210ba4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:19.606180   840 tablet_service.cc:1511] Processing CreateTablet for tablet 3b871863269841f7b0aa8baf5e640865 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:19.606863   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee81d93e84c4c689b02deb555210ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.607476   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b871863269841f7b0aa8baf5e640865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.613552   845 tablet_service.cc:1511] Processing CreateTablet for tablet c72e2f13b3404583a1c34e15098c42b9 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:19.614156   843 tablet_service.cc:1511] Processing CreateTablet for tablet d3d5a097d0fe49bab1a3ec1f78f4164d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:19.614904   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72e2f13b3404583a1c34e15098c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.621927   837 tablet_service.cc:1511] Processing CreateTablet for tablet b8c347b6586f486192f48c46974a2526 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:19.622958   837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c347b6586f486192f48c46974a2526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.625165   994 tablet_service.cc:1511] Processing CreateTablet for tablet f379c9f3a933409ca08bceb325f43132 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:19.626448   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379c9f3a933409ca08bceb325f43132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.630654   916 tablet_service.cc:1511] Processing CreateTablet for tablet 9155790402004e4f930983841dfd8c64 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:19.630976   845 tablet_service.cc:1511] Processing CreateTablet for tablet c3247a08a45747539334328f4abf2acc (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:19.632975   987 tablet_service.cc:1511] Processing CreateTablet for tablet 2b38ec4f420f4de9883306887a006c24 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:19.687085   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b38ec4f420f4de9883306887a006c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.634842   989 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed094558dad4fb2b45bc747bbc6adba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:12:19.692359   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed094558dad4fb2b45bc747bbc6adba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.637128   836 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee81d93e84c4c689b02deb555210ba4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:19.638753   842 tablet_service.cc:1511] Processing CreateTablet for tablet d548ef2981894fe6972da35c997f5cff (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:19.641081   993 tablet_service.cc:1511] Processing CreateTablet for tablet 701b27559f8e47f9b63e5d220d3d5a58 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:19.644129   920 tablet_service.cc:1511] Processing CreateTablet for tablet d548ef2981894fe6972da35c997f5cff (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:19.648715   914 tablet_service.cc:1511] Processing CreateTablet for tablet 3b871863269841f7b0aa8baf5e640865 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:19.650205   991 tablet_service.cc:1511] Processing CreateTablet for tablet a4018887e34a40108555fd3cbaa2c975 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:19.647428   840 tablet_service.cc:1511] Processing CreateTablet for tablet f379c9f3a933409ca08bceb325f43132 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:19.652619   841 tablet_service.cc:1511] Processing CreateTablet for tablet 2b38ec4f420f4de9883306887a006c24 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:19.697151   920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d548ef2981894fe6972da35c997f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.659931   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d5a097d0fe49bab1a3ec1f78f4164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.663290   912 tablet_service.cc:1511] Processing CreateTablet for tablet c72e2f13b3404583a1c34e15098c42b9 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:19.698452   836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee81d93e84c4c689b02deb555210ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.664565  1044 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:19.699689  1044 ts_tablet_manager.cc:1403] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.266s	user 0.011s	sys 0.004s
I20260709 13:12:19.664346   985 tablet_service.cc:1511] Processing CreateTablet for tablet c3247a08a45747539334328f4abf2acc (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:19.666625   844 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed094558dad4fb2b45bc747bbc6adba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:12:19.696468   993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701b27559f8e47f9b63e5d220d3d5a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.705917   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed094558dad4fb2b45bc747bbc6adba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.707269   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b38ec4f420f4de9883306887a006c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.708149   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3247a08a45747539334328f4abf2acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.711886   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d548ef2981894fe6972da35c997f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.712347   991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4018887e34a40108555fd3cbaa2c975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.714699   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379c9f3a933409ca08bceb325f43132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.666848   994 tablet_service.cc:1511] Processing CreateTablet for tablet 633fa6f0c6434c49aeb87667c9d97141 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:19.671355   917 tablet_service.cc:1511] Processing CreateTablet for tablet b8c347b6586f486192f48c46974a2526 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:19.676159   913 tablet_service.cc:1511] Processing CreateTablet for tablet c3247a08a45747539334328f4abf2acc (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:19.679183   915 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee81d93e84c4c689b02deb555210ba4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:19.717258   992 tablet_service.cc:1511] Processing CreateTablet for tablet c43ab43025004edcacdb3d8a30106319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:12:19.718394   988 tablet_service.cc:1511] Processing CreateTablet for tablet 53d4a96b7f094cf691b31dbe6fb015bd (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:19.718923   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72e2f13b3404583a1c34e15098c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.680869   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:19.720898   967 leader_election.cc:343] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:19.721359   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:19.722435  1049 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:19.723724   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3247a08a45747539334328f4abf2acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.726475   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b871863269841f7b0aa8baf5e640865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.729107   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3247a08a45747539334328f4abf2acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.682644  1045 tablet_bootstrap.cc:654] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.661295   911 tablet_service.cc:1511] Processing CreateTablet for tablet d3d5a097d0fe49bab1a3ec1f78f4164d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:19.729700   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d4a96b7f094cf691b31dbe6fb015bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.730317   992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43ab43025004edcacdb3d8a30106319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.730482   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d5a097d0fe49bab1a3ec1f78f4164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.736855   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee81d93e84c4c689b02deb555210ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.740139   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c347b6586f486192f48c46974a2526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.702992  1044 raft_consensus.cc:359] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.743908  1044 raft_consensus.cc:385] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:19.744257  1044 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:19.745025  1044 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.747174  1044 ts_tablet_manager.cc:1434] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.000s
I20260709 13:12:19.752030   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155790402004e4f930983841dfd8c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.756368   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 633fa6f0c6434c49aeb87667c9d97141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.769287  1046 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:19.786222  1046 ts_tablet_manager.cc:1403] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.630s	user 0.036s	sys 0.037s
I20260709 13:12:19.784013   990 tablet_service.cc:1511] Processing CreateTablet for tablet 248dca75dd144af2a2106e703fe92de4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:19.776342   919 tablet_service.cc:1511] Processing CreateTablet for tablet f379c9f3a933409ca08bceb325f43132 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:19.788772   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248dca75dd144af2a2106e703fe92de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.800800   916 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed094558dad4fb2b45bc747bbc6adba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:12:19.802032   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed094558dad4fb2b45bc747bbc6adba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.803910   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379c9f3a933409ca08bceb325f43132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.789214  1046 raft_consensus.cc:359] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.783699  1044 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:19.807792  1046 raft_consensus.cc:385] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:19.808197  1046 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:19.808399   838 tablet_service.cc:1511] Processing CreateTablet for tablet 701b27559f8e47f9b63e5d220d3d5a58 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:19.809098  1046 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.796710   839 tablet_service.cc:1511] Processing CreateTablet for tablet a4018887e34a40108555fd3cbaa2c975 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:19.809643   838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701b27559f8e47f9b63e5d220d3d5a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.816730   988 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3b97a883a4aeb97bdcc43b82a2b95 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:19.805203   840 tablet_service.cc:1511] Processing CreateTablet for tablet 633fa6f0c6434c49aeb87667c9d97141 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:19.793095   913 tablet_service.cc:1511] Processing CreateTablet for tablet 2b38ec4f420f4de9883306887a006c24 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:19.815099  1044 tablet_bootstrap.cc:654] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:19.818048   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3b97a883a4aeb97bdcc43b82a2b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.822995   915 tablet_service.cc:1511] Processing CreateTablet for tablet a4018887e34a40108555fd3cbaa2c975 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:19.824486   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4018887e34a40108555fd3cbaa2c975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.828815   844 tablet_service.cc:1511] Processing CreateTablet for tablet c43ab43025004edcacdb3d8a30106319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:12:19.818048   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 633fa6f0c6434c49aeb87667c9d97141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.840780   920 tablet_service.cc:1511] Processing CreateTablet for tablet 633fa6f0c6434c49aeb87667c9d97141 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:19.843734   920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 633fa6f0c6434c49aeb87667c9d97141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.835405   914 tablet_service.cc:1511] Processing CreateTablet for tablet 701b27559f8e47f9b63e5d220d3d5a58 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:19.844983   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701b27559f8e47f9b63e5d220d3d5a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.848341   912 tablet_service.cc:1511] Processing CreateTablet for tablet c43ab43025004edcacdb3d8a30106319 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:12:19.851845   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43ab43025004edcacdb3d8a30106319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.853055   987 tablet_service.cc:1511] Processing CreateTablet for tablet cfd817716cdc4448b3f884eecc94351c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:19.854471   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd817716cdc4448b3f884eecc94351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.855269   841 tablet_service.cc:1511] Processing CreateTablet for tablet cfd817716cdc4448b3f884eecc94351c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:19.856551   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b38ec4f420f4de9883306887a006c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.857519   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4018887e34a40108555fd3cbaa2c975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.876159   911 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3b97a883a4aeb97bdcc43b82a2b95 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:19.877315   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43ab43025004edcacdb3d8a30106319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.864830   836 tablet_service.cc:1511] Processing CreateTablet for tablet 248dca75dd144af2a2106e703fe92de4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:19.868350   918 tablet_service.cc:1511] Processing CreateTablet for tablet 248dca75dd144af2a2106e703fe92de4 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:19.868546   845 tablet_service.cc:1511] Processing CreateTablet for tablet 95c3b97a883a4aeb97bdcc43b82a2b95 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:19.872291   842 tablet_service.cc:1511] Processing CreateTablet for tablet 53d4a96b7f094cf691b31dbe6fb015bd (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:19.857276   917 tablet_service.cc:1511] Processing CreateTablet for tablet 53d4a96b7f094cf691b31dbe6fb015bd (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:19.877625   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3b97a883a4aeb97bdcc43b82a2b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.889196   836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248dca75dd144af2a2106e703fe92de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.890431  1046 ts_tablet_manager.cc:1434] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.104s	user 0.001s	sys 0.015s
I20260709 13:12:19.891067   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d4a96b7f094cf691b31dbe6fb015bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.893946   989 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec15122641447a3bd48017eb990578d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:19.896521   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec15122641447a3bd48017eb990578d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.904965  1054 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.905563  1054 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.912065  1054 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:19.926839   916 tablet_service.cc:1511] Processing CreateTablet for tablet cfd817716cdc4448b3f884eecc94351c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:19.933583  1045 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:19.934165  1045 ts_tablet_manager.cc:1403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.390s	user 0.018s	sys 0.004s
I20260709 13:12:19.934665  1044 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:19.935329  1044 ts_tablet_manager.cc:1403] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.152s	user 0.018s	sys 0.000s
I20260709 13:12:19.936777   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248dca75dd144af2a2106e703fe92de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.938691  1044 raft_consensus.cc:359] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:19.940191   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d4a96b7f094cf691b31dbe6fb015bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.940574   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95c3b97a883a4aeb97bdcc43b82a2b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.942001   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd817716cdc4448b3f884eecc94351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.927454  1046 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:19.952534   919 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec15122641447a3bd48017eb990578d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:19.954433   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec15122641447a3bd48017eb990578d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.897332   837 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec15122641447a3bd48017eb990578d (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:19.957774   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd817716cdc4448b3f884eecc94351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.957847   837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec15122641447a3bd48017eb990578d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.967736  1044 raft_consensus.cc:385] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.011024  1045 raft_consensus.cc:359] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.011935  1045 raft_consensus.cc:385] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.012287  1045 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.013054  1045 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.021804   985 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4613bf5fb0446a80ac401fe0560967 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:20.023272   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4613bf5fb0446a80ac401fe0560967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:19.985213  1044 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.024176  1044 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.024824   987 tablet_service.cc:1511] Processing CreateTablet for tablet 90a3708334cc4bd59c73d494d2747d54 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:20.026682  1044 ts_tablet_manager.cc:1434] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.091s	user 0.002s	sys 0.004s
I20260709 13:12:20.032641  1045 ts_tablet_manager.cc:1434] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.004s
I20260709 13:12:20.033785  1045 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.034403   987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a3708334cc4bd59c73d494d2747d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.041730   913 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4613bf5fb0446a80ac401fe0560967 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:20.043278   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4613bf5fb0446a80ac401fe0560967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.054734   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:20.056509   818 leader_election.cc:343] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.057569  1044 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:20.060068  1046 tablet_bootstrap.cc:654] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.062245   992 tablet_service.cc:1511] Processing CreateTablet for tablet 24f44707b1d3466e91403f063187a9b2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:20.064597   992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f44707b1d3466e91403f063187a9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.070216   915 tablet_service.cc:1511] Processing CreateTablet for tablet 90a3708334cc4bd59c73d494d2747d54 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:20.094576   989 tablet_service.cc:1511] Processing CreateTablet for tablet 6444551c84374ec98f4becf8b07446ba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:20.096619  1044 tablet_bootstrap.cc:654] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.096649   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6444551c84374ec98f4becf8b07446ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.097193   842 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4613bf5fb0446a80ac401fe0560967 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:20.109256   988 tablet_service.cc:1511] Processing CreateTablet for tablet 2840f4bbf21443318b9ad41dc7e3e853 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:20.110672   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2840f4bbf21443318b9ad41dc7e3e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.110702  1057 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.112283  1057 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.090694   915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a3708334cc4bd59c73d494d2747d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.119143  1057 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.132148   842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4613bf5fb0446a80ac401fe0560967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.153760   843 tablet_service.cc:1511] Processing CreateTablet for tablet 2840f4bbf21443318b9ad41dc7e3e853 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:20.145637   845 tablet_service.cc:1511] Processing CreateTablet for tablet 90a3708334cc4bd59c73d494d2747d54 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:20.145651   836 tablet_service.cc:1511] Processing CreateTablet for tablet 24f44707b1d3466e91403f063187a9b2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:20.157249   839 tablet_service.cc:1511] Processing CreateTablet for tablet 6444551c84374ec98f4becf8b07446ba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:20.159977   994 tablet_service.cc:1511] Processing CreateTablet for tablet 516d927af6bf46aa947c0919f1dfc830 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:20.161377   912 tablet_service.cc:1511] Processing CreateTablet for tablet 6444551c84374ec98f4becf8b07446ba (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:20.161710   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516d927af6bf46aa947c0919f1dfc830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.162773   912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6444551c84374ec98f4becf8b07446ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.171245   845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a3708334cc4bd59c73d494d2747d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.172375   919 tablet_service.cc:1511] Processing CreateTablet for tablet 2840f4bbf21443318b9ad41dc7e3e853 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:20.171314   911 tablet_service.cc:1511] Processing CreateTablet for tablet 516d927af6bf46aa947c0919f1dfc830 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:20.173338   836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f44707b1d3466e91403f063187a9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.173636   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2840f4bbf21443318b9ad41dc7e3e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.175971   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6444551c84374ec98f4becf8b07446ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.185240  1049 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.185858  1049 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.189420   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2840f4bbf21443318b9ad41dc7e3e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.189463   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516d927af6bf46aa947c0919f1dfc830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.168571   916 tablet_service.cc:1511] Processing CreateTablet for tablet 24f44707b1d3466e91403f063187a9b2 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:20.195797   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f44707b1d3466e91403f063187a9b2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:20.228523   784 debug-util.cc:398] Leaking SignalData structure 0x7b080003d980 after lost signal to thread 32227
W20260709 13:12:20.229494   784 debug-util.cc:398] Leaking SignalData structure 0x7b080003e060 after lost signal to thread 787
I20260709 13:12:20.230775  1044 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
W20260709 13:12:20.243907   740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44651.7us
I20260709 13:12:20.244357  1046 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.244845  1046 ts_tablet_manager.cc:1403] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.318s	user 0.019s	sys 0.050s
I20260709 13:12:20.245443   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:20.246907   967 leader_election.cc:343] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.249280   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:20.250905   969 leader_election.cc:343] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.251384   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.252329   914 tablet_service.cc:1511] Processing CreateTablet for tablet 14a8781662d54452a1478c2ae0e3f911 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:20.253669   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a8781662d54452a1478c2ae0e3f911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.231410  1044 ts_tablet_manager.cc:1403] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.204s	user 0.024s	sys 0.042s
I20260709 13:12:20.267443  1046 raft_consensus.cc:359] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.268246  1046 raft_consensus.cc:385] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.268635  1046 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.269388  1046 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.270828  1045 tablet_bootstrap.cc:654] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.270756   985 tablet_service.cc:1511] Processing CreateTablet for tablet 14a8781662d54452a1478c2ae0e3f911 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:20.273100   990 tablet_service.cc:1511] Processing CreateTablet for tablet e0cca622722b48a7ba611a3b77376f80 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:20.268311  1044 raft_consensus.cc:359] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.283201   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a8781662d54452a1478c2ae0e3f911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.298964   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:20.284691  1054 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.300155  1054 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:20.300698   895 leader_election.cc:343] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.285893  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.302270  1054 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:20.302528   821 leader_election.cc:343] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.302953   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.303223   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0cca622722b48a7ba611a3b77376f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.286703   918 tablet_service.cc:1511] Processing CreateTablet for tablet e0cca622722b48a7ba611a3b77376f80 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:20.287259   844 tablet_service.cc:1511] Processing CreateTablet for tablet ad6ca2a4a8134a10b60d7308b9a660db (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:12:20.288841   838 tablet_service.cc:1511] Processing CreateTablet for tablet e0cca622722b48a7ba611a3b77376f80 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:20.281643  1049 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:20.290002   841 tablet_service.cc:1511] Processing CreateTablet for tablet 14a8781662d54452a1478c2ae0e3f911 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:20.335891  1049 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.290850   840 tablet_service.cc:1511] Processing CreateTablet for tablet 516d927af6bf46aa947c0919f1dfc830 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:20.292814  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.337514  1003 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:20.339260   896 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:20.340696  1057 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.341142  1057 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.341573  1057 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.303987   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:20.305018  1054 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.317729   911 tablet_service.cc:1511] Processing CreateTablet for tablet ad6ca2a4a8134a10b60d7308b9a660db (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:12:20.325160  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:20.345441   818 leader_election.cc:343] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.332247   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6ca2a4a8134a10b60d7308b9a660db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.334197   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0cca622722b48a7ba611a3b77376f80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:20.349718   821 leader_election.cc:343] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.305814  1044 raft_consensus.cc:385] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.346877  1046 ts_tablet_manager.cc:1434] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 13:12:20.353155   821 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.357993  1054 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.359486   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6ca2a4a8134a10b60d7308b9a660db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.360357   839 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfb3f7cad745918e1bc1faf5cc697c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:20.360834  1057 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.365820   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:20.366335  1057 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:20.366974   895 leader_election.cc:343] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.367801   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a8781662d54452a1478c2ae0e3f911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.367820  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:20.368587  1004 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.369589   989 tablet_service.cc:1511] Processing CreateTablet for tablet ad6ca2a4a8134a10b60d7308b9a660db (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 13:12:20.355152   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 104280us
I20260709 13:12:20.377000  1004 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:20.379462   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6ca2a4a8134a10b60d7308b9a660db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.380182  1045 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:20.384171  1045 ts_tablet_manager.cc:1403] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.351s	user 0.014s	sys 0.024s
I20260709 13:12:20.387277  1045 raft_consensus.cc:359] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.352583  1044 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.380649   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516d927af6bf46aa947c0919f1dfc830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.401383   838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0cca622722b48a7ba611a3b77376f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.402158   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfb3f7cad745918e1bc1faf5cc697c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.415843  1044 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.388470  1046 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.388825   896 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:12:20.389693   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:20.392807   919 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfb3f7cad745918e1bc1faf5cc697c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:20.420266  1057 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:20.421115   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfb3f7cad745918e1bc1faf5cc697c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.425118  1044 ts_tablet_manager.cc:1434] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.159s	user 0.007s	sys 0.000s
I20260709 13:12:20.426272  1044 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:20.394465   994 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfb3f7cad745918e1bc1faf5cc697c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:20.428388   994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfb3f7cad745918e1bc1faf5cc697c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.399962  1045 raft_consensus.cc:385] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.435978  1045 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.436790  1045 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.438864  1045 ts_tablet_manager.cc:1434] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.002s
I20260709 13:12:20.442342   843 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4d7e87915947148833db3fe84cf623 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:20.443652   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4d7e87915947148833db3fe84cf623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.444407  1054 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.445039  1054 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.446162  1057 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:20.447329  1054 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.454622  1045 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.447014  1057 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.448410  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.457559  1004 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:20.448941   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:20.459400   818 leader_election.cc:343] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.460134   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.461663  1054 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.462203  1054 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.462714  1054 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.467073   989 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4d7e87915947148833db3fe84cf623 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:20.468289   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4d7e87915947148833db3fe84cf623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.468803   916 tablet_service.cc:1511] Processing CreateTablet for tablet 82ed8549f06d4ef3a4620f596e0139b7 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:20.471434   840 tablet_service.cc:1511] Processing CreateTablet for tablet 82ed8549f06d4ef3a4620f596e0139b7 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:20.472651   840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ed8549f06d4ef3a4620f596e0139b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.476071  1068 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.476581  1068 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.481249  1070 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.481877  1070 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.485051   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:20.486538   895 leader_election.cc:343] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.488515   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:20.450973  1046 tablet_bootstrap.cc:654] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.449545   918 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4d7e87915947148833db3fe84cf623 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:20.488639  1068 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:20.490020   969 leader_election.cc:343] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.490949  1044 tablet_bootstrap.cc:654] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.491878  1045 tablet_bootstrap.cc:654] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.492972  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.493649  1004 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:20.495002   896 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:20.495873  1070 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.496450  1070 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.496829  1070 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.497287  1070 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.500221   988 tablet_service.cc:1511] Processing CreateTablet for tablet 82ed8549f06d4ef3a4620f596e0139b7 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:20.504566   988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ed8549f06d4ef3a4620f596e0139b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.505762  1054 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.511373   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:20.512171   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4d7e87915947148833db3fe84cf623. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:20.512684   967 leader_election.cc:343] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.513120   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.513856  1068 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.514437  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:20.514971  1004 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.527758   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ed8549f06d4ef3a4620f596e0139b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.529266  1044 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:20.529971  1044 ts_tablet_manager.cc:1403] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.001s
I20260709 13:12:20.533972   911 tablet_service.cc:1511] Processing CreateTablet for tablet dcb3d540c3fc49bf926540f7bd349110 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:20.536728   911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb3d540c3fc49bf926540f7bd349110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.536319   753 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:20.543483  1070 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.550776   989 tablet_service.cc:1511] Processing CreateTablet for tablet dcb3d540c3fc49bf926540f7bd349110 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:20.552317   989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb3d540c3fc49bf926540f7bd349110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.516892  1054 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.523036  1004 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:20.560950   919 tablet_service.cc:1511] Processing CreateTablet for tablet 6533a74f2a524baab75bdfae9b5b1888 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:20.561175   844 tablet_service.cc:1511] Processing CreateTablet for tablet 6533a74f2a524baab75bdfae9b5b1888 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:20.569809   844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6533a74f2a524baab75bdfae9b5b1888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.570768   990 tablet_service.cc:1511] Processing CreateTablet for tablet 6533a74f2a524baab75bdfae9b5b1888 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:20.572103   990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6533a74f2a524baab75bdfae9b5b1888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.574914   914 tablet_service.cc:1511] Processing CreateTablet for tablet 31088ed18ff54bcea05d848f8c67e437 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:20.576249   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31088ed18ff54bcea05d848f8c67e437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.576161   839 tablet_service.cc:1511] Processing CreateTablet for tablet dcb3d540c3fc49bf926540f7bd349110 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:20.577440   839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb3d540c3fc49bf926540f7bd349110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.577293   985 tablet_service.cc:1511] Processing CreateTablet for tablet 31088ed18ff54bcea05d848f8c67e437 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:20.578501   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31088ed18ff54bcea05d848f8c67e437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.592284   919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6533a74f2a524baab75bdfae9b5b1888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.593024   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:20.594350  1054 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:20.596602   843 tablet_service.cc:1511] Processing CreateTablet for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:20.597803   843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.600732  1044 raft_consensus.cc:359] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.601585  1044 raft_consensus.cc:385] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.601516   985 tablet_service.cc:1511] Processing CreateTablet for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:20.602015  1044 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.602475   913 tablet_service.cc:1511] Processing CreateTablet for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:20.602907   985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.605556   913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.618197   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:20.618927  1046 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.619434  1046 ts_tablet_manager.cc:1403] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.231s	user 0.013s	sys 0.009s
W20260709 13:12:20.621107   895 leader_election.cc:343] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.621588   841 tablet_service.cc:1511] Processing CreateTablet for tablet 31088ed18ff54bcea05d848f8c67e437 (DEFAULT_TABLE table=test-workload [id=503712c4318e4a7b80efce944ee5995b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:20.622882   841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31088ed18ff54bcea05d848f8c67e437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:20.624636  1045 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:20.635883  1045 ts_tablet_manager.cc:1403] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.181s	user 0.004s	sys 0.024s
I20260709 13:12:20.622435  1046 raft_consensus.cc:359] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:20.636483   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 141596us
I20260709 13:12:20.610023  1044 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.641010  1046 raft_consensus.cc:385] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.643661  1046 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.639711  1054 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:20.648957  1044 ts_tablet_manager.cc:1434] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.119s	user 0.010s	sys 0.000s
I20260709 13:12:20.670099  1045 raft_consensus.cc:359] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.675181  1045 raft_consensus.cc:385] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.675473  1045 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.644419  1046 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.641568  1070 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.676189  1045 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.672264  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:20.677974  1004 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.673004   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:20.679931  1046 ts_tablet_manager.cc:1434] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:12:20.680092  1045 ts_tablet_manager.cc:1434] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.001s
I20260709 13:12:20.692595  1046 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.692691  1045 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.694921  1070 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:20.682406   818 leader_election.cc:343] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.685310  1004 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:20.686317  1044 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:20.695474  1070 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.686990  1054 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.700246  1045 tablet_bootstrap.cc:654] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.700132   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:20.701472   895 leader_election.cc:343] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.702255   896 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:20.702378  1070 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.707242  1044 tablet_bootstrap.cc:654] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.708724  1070 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:20.709260  1070 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:20.710117  1070 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.712466  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.708711  1046 tablet_bootstrap.cc:654] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:20.714092   896 leader_election.cc:343] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.714578   896 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:20.715488  1075 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.730068  1044 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:20.730773  1044 ts_tablet_manager.cc:1403] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.002s
I20260709 13:12:20.739101  1046 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.739886  1046 ts_tablet_manager.cc:1403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.018s
I20260709 13:12:20.744088  1046 raft_consensus.cc:359] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.745102  1046 raft_consensus.cc:385] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.745512  1046 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.746670  1046 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.747548  1045 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:20.748117  1045 ts_tablet_manager.cc:1403] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.002s
I20260709 13:12:20.749892  1046 ts_tablet_manager.cc:1434] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:12:20.750960  1046 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.734369  1044 raft_consensus.cc:359] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.751145  1045 raft_consensus.cc:359] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.755905  1044 raft_consensus.cc:385] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.756000  1045 raft_consensus.cc:385] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.756305  1044 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.756438  1045 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.756916  1046 tablet_bootstrap.cc:654] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.757265  1045 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.751776   751 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:20.762156  1045 ts_tablet_manager.cc:1434] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:12:20.763234  1045 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.770123  1045 tablet_bootstrap.cc:654] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.774819   753 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:20.778461  1068 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.779162  1068 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.783787   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:20.787142   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:20.789536   969 leader_election.cc:343] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.790397  1068 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:20.791215   967 leader_election.cc:343] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.792061   967 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.793568  1068 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.796051  1045 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:20.796552  1045 ts_tablet_manager.cc:1403] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.004s
I20260709 13:12:20.799561  1045 raft_consensus.cc:359] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.800487  1045 raft_consensus.cc:385] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.800920  1045 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.757169  1044 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.802284  1045 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.809980  1044 ts_tablet_manager.cc:1434] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.001s
I20260709 13:12:20.816025  1044 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:20.826254  1045 ts_tablet_manager.cc:1434] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:12:20.827113  1044 tablet_bootstrap.cc:654] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.831552  1046 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.831835  1045 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.832088  1046 ts_tablet_manager.cc:1403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.081s	user 0.047s	sys 0.016s
I20260709 13:12:20.835059  1046 raft_consensus.cc:359] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.835930  1046 raft_consensus.cc:385] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.837716  1046 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.838527  1046 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.842469  1046 ts_tablet_manager.cc:1434] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:12:20.843703  1046 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.850194  1046 tablet_bootstrap.cc:654] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.851230  1045 tablet_bootstrap.cc:654] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.854697  1070 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.855218  1070 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.857416  1070 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.859035  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.860363   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:20.861824   895 leader_election.cc:343] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.862356  1044 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:20.862869  1044 ts_tablet_manager.cc:1403] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.004s
W20260709 13:12:20.863019   896 leader_election.cc:343] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.863443   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:20.864545  1070 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:20.865866  1044 raft_consensus.cc:359] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.866586  1044 raft_consensus.cc:385] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.866876  1044 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.867740  1044 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.874972  1045 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:20.875630  1045 ts_tablet_manager.cc:1403] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.009s
I20260709 13:12:20.878738  1045 raft_consensus.cc:359] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.879580  1045 raft_consensus.cc:385] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.879943  1045 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.880872  1045 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.881313  1044 ts_tablet_manager.cc:1434] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:12:20.885416  1044 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:20.886168  1068 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.886804  1046 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.886672  1068 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.887121  1045 ts_tablet_manager.cc:1434] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:12:20.887320  1046 ts_tablet_manager.cc:1403] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.044s	user 0.021s	sys 0.004s
I20260709 13:12:20.888882  1068 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:20.890355  1046 raft_consensus.cc:359] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.891093  1046 raft_consensus.cc:385] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.891400  1046 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.892186  1046 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.893677   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:20.894402  1046 ts_tablet_manager.cc:1434] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:12:20.894304   855 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:20.895592   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:20.896731  1044 tablet_bootstrap.cc:654] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.896929   855 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.898483  1068 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.898895  1068 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.899233  1068 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.902338   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:20.904315   967 leader_election.cc:343] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.905460  1046 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.910857  1045 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:20.917595  1075 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:20.921638  1046 tablet_bootstrap.cc:654] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.929705  1068 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.933660   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:20.934255  1068 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:20.935173   967 leader_election.cc:343] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.935508  1044 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:20.936056   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:20.936782   855 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.936246  1044 ts_tablet_manager.cc:1403] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.010s
I20260709 13:12:20.942387  1044 raft_consensus.cc:359] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.943120  1044 raft_consensus.cc:385] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.943414  1044 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.948294  1045 tablet_bootstrap.cc:654] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.951943  1075 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.952494  1075 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.954727  1046 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:20.954777  1075 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.955435  1046 ts_tablet_manager.cc:1403] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.006s
I20260709 13:12:20.955969  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.957129   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:20.957775   853 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:20.958602  1046 raft_consensus.cc:359] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:20.959028   896 leader_election.cc:343] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.959424  1046 raft_consensus.cc:385] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:20.959841  1046 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:20.960184   895 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:20.961046  1070 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.960676  1046 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.961417  1070 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.961792  1070 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.970072  1054 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.970649  1054 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.972931  1054 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.973551  1046 ts_tablet_manager.cc:1434] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.011s
I20260709 13:12:20.974046  1054 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.974684  1046 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:20.974553  1054 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.974081  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:20.975906  1004 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:20.977180  1054 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:20.979102   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:20.974792   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:20.981544   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:20.981900  1046 tablet_bootstrap.cc:654] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:20.983068  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:20.987890   818 leader_election.cc:343] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:20.988593   818 leader_election.cc:343] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:20.989995  1077 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:20.990515  1077 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.990881  1077 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:20.990584  1099 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:20.993111  1044 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:20.993981  1003 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:20.997776   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:20.998696  1044 ts_tablet_manager.cc:1434] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 13:12:20.999750  1044 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:21.005085  1046 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.005666  1046 ts_tablet_manager.cc:1403] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.006s
I20260709 13:12:21.007047   855 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:21.008368  1046 raft_consensus.cc:359] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.009187  1046 raft_consensus.cc:385] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.009542  1046 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.010402  1046 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.011968  1054 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.012328  1054 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.012662  1054 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.012441   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:21.013906  1068 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.015874  1046 ts_tablet_manager.cc:1434] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:21.016826  1046 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.018493  1070 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.021613  1045 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.022071  1045 ts_tablet_manager.cc:1403] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.111s	user 0.008s	sys 0.003s
I20260709 13:12:21.022485  1046 tablet_bootstrap.cc:654] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.024521  1077 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.028093   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:21.029672   818 leader_election.cc:343] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.034518  1046 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.035130  1046 ts_tablet_manager.cc:1403] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.009s
I20260709 13:12:21.037761  1044 tablet_bootstrap.cc:654] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.037964  1046 raft_consensus.cc:359] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.038872  1046 raft_consensus.cc:385] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.039181  1046 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.040097  1046 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.041611  1068 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:21.042219  1046 ts_tablet_manager.cc:1434] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:21.042498  1068 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.043426  1046 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.049538  1046 tablet_bootstrap.cc:654] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.050486   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:21.050712  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.051192   855 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:21.051874   896 leader_election.cc:343] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.054454  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.055094  1004 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.056021  1070 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.056707  1077 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.055203  1045 raft_consensus.cc:359] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.057837  1045 raft_consensus.cc:385] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.058198  1045 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.060523  1054 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.064174   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:21.064648   930 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.066143  1046 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.066669  1046 ts_tablet_manager.cc:1403] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.006s
I20260709 13:12:21.069222  1046 raft_consensus.cc:359] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.069944  1046 raft_consensus.cc:385] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.070282  1046 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.071827   930 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.071076  1046 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.073091   818 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:21.073993  1046 ts_tablet_manager.cc:1434] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:12:21.074836  1046 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.080879  1046 tablet_bootstrap.cc:654] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.081482  1099 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.081990  1099 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.084056   855 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:21.084255  1099 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.085326   895 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:21.086756  1099 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.087220  1099 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:21.087925   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.088713   853 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:21.088333  1099 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.088357  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.087224  1077 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.093449  1077 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:21.094412   753 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.096029   896 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.096747  1103 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:21.094507  1077 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.098193  1045 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.108489  1070 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.114220  1045 ts_tablet_manager.cc:1434] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.092s	user 0.009s	sys 0.000s
I20260709 13:12:21.115303  1045 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.120285  1112 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.120788  1112 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.124920  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.125049   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:21.126937   818 leader_election.cc:343] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:21.129930   821 leader_election.cc:343] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.130388   821 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:21.131102  1103 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.131330  1070 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.132055  1112 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.132249  1054 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.109375  1004 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.134068  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.134639  1004 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.140021   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:21.141817  1045 tablet_bootstrap.cc:654] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.154238   751 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.157496  1044 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:21.158043  1044 ts_tablet_manager.cc:1403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.004s
I20260709 13:12:21.167104  1112 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.168507  1112 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:21.174968   753 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.185504   930 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.181984  1046 tablet_bootstrap.cc:492] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.188714  1046 ts_tablet_manager.cc:1403] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.114s	user 0.037s	sys 0.055s
I20260709 13:12:21.191340  1046 raft_consensus.cc:359] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.192034  1046 raft_consensus.cc:385] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.192308  1046 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.193418  1112 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.183635  1044 raft_consensus.cc:359] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.196923  1044 raft_consensus.cc:385] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.197221  1044 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.197872  1044 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.199345  1004 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.199784  1044 ts_tablet_manager.cc:1434] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.003s
I20260709 13:12:21.200042  1045 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.200484  1046 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.200503  1045 ts_tablet_manager.cc:1403] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.004s
I20260709 13:12:21.204061  1045 raft_consensus.cc:359] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.207144  1077 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.207899  1045 raft_consensus.cc:385] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.208323  1045 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.209425  1045 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.213712  1046 ts_tablet_manager.cc:1434] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:12:21.214706  1046 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.216311  1044 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:21.219954  1045 ts_tablet_manager.cc:1434] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:12:21.221306  1045 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.229297  1045 tablet_bootstrap.cc:654] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.234961   753 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.252664  1112 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.253243  1112 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.259997  1112 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.260746   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.261449   929 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:21.262770   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:21.263711  1112 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:21.263942   895 consensus_peers.cc:597] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:21.264133  1112 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.269776  1112 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.264101  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.270851  1003 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:21.274040  1046 tablet_bootstrap.cc:654] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.277254  1044 tablet_bootstrap.cc:654] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.290411  1112 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.294222  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.294297   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:21.294901  1003 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.295224   929 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.301648  1099 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:21.303618  1003 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.305081   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:21.306016  1103 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.306627  1103 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:21.307564  1103 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.316053  1112 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.340278  1070 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.341025  1070 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.343688  1070 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.346426   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:21.348284   895 leader_election.cc:343] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.346511  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:21.350278   896 leader_election.cc:343] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.350703   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.357813  1045 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.358536  1045 ts_tablet_manager.cc:1403] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.003s
I20260709 13:12:21.351528  1070 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.364351  1068 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.365222  1044 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:21.365041  1068 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.365688  1044 ts_tablet_manager.cc:1403] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.150s	user 0.023s	sys 0.042s
I20260709 13:12:21.378047   929 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.386229   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.386803  1068 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:21.387801   969 leader_election.cc:343] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.388718   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.389353   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:21.390527   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.391747  1068 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.392168  1068 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.392565  1068 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.388487  1045 raft_consensus.cc:359] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.399842  1045 raft_consensus.cc:385] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.400192  1045 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.401386  1046 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.401024  1045 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.400779   752 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.401918  1046 ts_tablet_manager.cc:1403] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.187s	user 0.055s	sys 0.046s
I20260709 13:12:21.403117  1044 raft_consensus.cc:359] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.403771  1044 raft_consensus.cc:385] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.404042  1044 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.404703  1044 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.405071  1046 raft_consensus.cc:359] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.405879  1046 raft_consensus.cc:385] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.406204  1046 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.407021  1046 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.409373  1046 ts_tablet_manager.cc:1434] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:21.413165  1046 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.407444  1045 ts_tablet_manager.cc:1434] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:12:21.423116  1045 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.431202  1046 tablet_bootstrap.cc:654] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.426550  1068 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.427465  1044 ts_tablet_manager.cc:1434] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.000s
I20260709 13:12:21.436775  1044 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
W20260709 13:12:21.449152   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:21.453044  1046 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.453507  1046 ts_tablet_manager.cc:1403] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.000s
I20260709 13:12:21.458256  1044 tablet_bootstrap.cc:654] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.457947  1046 raft_consensus.cc:359] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.458632  1046 raft_consensus.cc:385] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.458909  1046 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.459509  1045 tablet_bootstrap.cc:654] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.459470  1046 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.461647  1046 ts_tablet_manager.cc:1434] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:21.462883  1068 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.464112   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:21.465602   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:21.466212   929 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:21.470743   969 leader_election.cc:343] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.474732   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:21.476437   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.477246  1068 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.477840  1068 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:21.478821  1068 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.481348  1045 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.481798  1045 ts_tablet_manager.cc:1403] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.003s
I20260709 13:12:21.477885  1046 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.484549  1045 raft_consensus.cc:359] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.485255  1045 raft_consensus.cc:385] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.485530  1045 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.489570  1070 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.490105  1070 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.492337  1070 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.492414   752 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.494449  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.494832  1046 tablet_bootstrap.cc:654] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.494867   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.495669   855 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
W20260709 13:12:21.495983   896 leader_election.cc:343] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.496968   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:21.498310  1070 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.498421  1099 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.498733  1070 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.499114  1070 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.498881  1099 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.500797  1045 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.506618  1070 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.510049  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.510380  1070 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.510912  1004 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:21.514318  1045 ts_tablet_manager.cc:1434] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:12:21.516019  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.523511  1045 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.525076   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:21.525959   853 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:21.529464   896 consensus_peers.cc:597] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:21.530200  1045 tablet_bootstrap.cc:654] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.531423   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.533437  1044 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:21.533893  1044 ts_tablet_manager.cc:1403] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.002s
I20260709 13:12:21.535986   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.536893  1097 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.536599  1044 raft_consensus.cc:359] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.537375  1044 raft_consensus.cc:385] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.537670  1044 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.507232  1099 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.538499  1044 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:21.530354   896 leader_election.cc:343] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.545782   853 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:21.552943   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:21.555801  1070 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.556357  1070 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:21.557248  1070 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.561105  1057 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.562660  1103 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.565675  1044 ts_tablet_manager.cc:1434] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:12:21.566882  1044 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:21.583698  1097 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.584586  1046 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.585089  1046 ts_tablet_manager.cc:1403] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.107s	user 0.031s	sys 0.013s
I20260709 13:12:21.589924  1045 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.590515  1045 ts_tablet_manager.cc:1403] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.022s
I20260709 13:12:21.592803  1046 raft_consensus.cc:359] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.593495  1046 raft_consensus.cc:385] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.593756  1046 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.597136   753 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.600349  1044 tablet_bootstrap.cc:654] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.600898  1045 raft_consensus.cc:359] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.601601  1045 raft_consensus.cc:385] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.601316  1046 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.601950  1045 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.605333  1046 ts_tablet_manager.cc:1434] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:12:21.605212  1045 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.606225  1046 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.606860  1103 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.612576  1046 tablet_bootstrap.cc:654] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.614033  1045 ts_tablet_manager.cc:1434] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:12:21.620211  1103 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.627723  1112 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.628218  1112 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.630594  1112 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.615154  1045 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.633884  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.634593  1002 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:21.637054   821 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:21.638091  1112 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.638469  1112 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.638818  1112 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.640293   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.640964   929 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:21.642279  1045 tablet_bootstrap.cc:654] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.644600  1044 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:21.645051  1044 ts_tablet_manager.cc:1403] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.079s	user 0.006s	sys 0.014s
I20260709 13:12:21.646433  1112 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.648073  1044 raft_consensus.cc:359] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.648700  1044 raft_consensus.cc:385] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.648967  1044 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.650931  1044 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.653447  1044 ts_tablet_manager.cc:1434] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:21.654498   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:21.654803  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.655097   929 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.655380  1002 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.655462  1103 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.655933  1103 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.657971  1103 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.660159  1044 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:21.659945   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.662993  1112 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.663092  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.665408  1046 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.665880  1046 ts_tablet_manager.cc:1403] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.007s
W20260709 13:12:21.668496   818 leader_election.cc:343] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.669202   929 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.668433  1046 raft_consensus.cc:359] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.669737  1046 raft_consensus.cc:385] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:21.670353   821 leader_election.cc:343] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.670610   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:21.670781   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:21.671443  1112 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.672000  1103 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.672128  1002 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:21.672780  1046 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.672978  1112 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:21.673708  1046 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.674052  1112 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.680073  1046 ts_tablet_manager.cc:1434] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:21.680948  1068 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:21.683897  1046 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.694931  1106 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.696005  1106 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.701797   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.702032  1046 tablet_bootstrap.cc:654] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.703446  1045 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:21.704136  1045 ts_tablet_manager.cc:1403] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.019s
W20260709 13:12:21.705036   969 leader_election.cc:343] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.705454  1106 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.708990   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.709600   930 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:21.714478   967 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.715709  1068 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.717011  1068 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.717366  1068 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.717675  1068 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.722890  1044 tablet_bootstrap.cc:654] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.731168  1045 raft_consensus.cc:359] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.732020  1045 raft_consensus.cc:385] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.732385  1045 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.733024  1045 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.740357  1054 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.743533  1068 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.745697  1054 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.754151  1068 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.754426   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:21.755290   928 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.752887   753 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.758896   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:21.760766   969 leader_election.cc:343] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.746187  1054 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.764204  1054 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.767055   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:21.773051  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.776827   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:21.780287  1103 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.790217  1045 ts_tablet_manager.cc:1434] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.086s	user 0.009s	sys 0.005s
I20260709 13:12:21.797521  1045 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:21.798106  1112 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:21.804649   928 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:21.806028   967 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.807247  1106 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:21.807927  1106 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:21.809011  1106 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.849292  1045 tablet_bootstrap.cc:654] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.856169  1044 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:21.856680  1044 ts_tablet_manager.cc:1403] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.203s	user 0.010s	sys 0.004s
I20260709 13:12:21.858584   853 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.859443  1044 raft_consensus.cc:359] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.860307  1044 raft_consensus.cc:385] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.860567  1044 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.862324  1044 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.861142  1097 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.862756  1099 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.864665  1097 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.864814  1099 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.865768  1044 ts_tablet_manager.cc:1434] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:12:21.865815  1046 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:21.866379  1046 ts_tablet_manager.cc:1403] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.183s	user 0.044s	sys 0.016s
I20260709 13:12:21.866736  1044 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:21.867244  1097 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.867106  1099 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.870941   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.872496   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.873123   854 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:21.875435  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:21.876151  1002 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:21.877053  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:21.879792   895 leader_election.cc:343] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.880029   896 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:21.881506  1097 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.881999  1097 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.882391  1097 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.887507  1145 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:12:21.889920  1145 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.890394  1145 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.905125  1140 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:12:21.910373   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.911270   854 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:21.914057   969 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:21.915241  1068 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.915858  1068 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.916299  1068 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.923195   751 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.924860  1106 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.925242  1046 raft_consensus.cc:359] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.925328  1106 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.925985  1046 raft_consensus.cc:385] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:21.926285  1046 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:21.927201  1046 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.928534  1106 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.929632  1046 ts_tablet_manager.cc:1434] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.063s	user 0.010s	sys 0.000s
I20260709 13:12:21.930624  1046 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:21.928964  1145 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.936259   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:21.937620   854 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
W20260709 13:12:21.937098   969 consensus_peers.cc:597] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:21.940369   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:21.941628  1106 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:21.942181  1106 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.942633  1106 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:21.935205   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46366.1us
I20260709 13:12:21.945478  1097 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.948937  1099 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.949591  1099 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.951766  1097 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:21.955567   896 leader_election.cc:343] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:21.961283   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:21.963073  1044 tablet_bootstrap.cc:654] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.959468  1099 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:21.966272  1057 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:21.969995  1046 tablet_bootstrap.cc:654] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:21.971735  1068 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:21.973173  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:21.973944  1004 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.974377   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:21.974932   854 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:21.978421  1068 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:21.998610   854 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:21.999910   969 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:22.002584  1068 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.003278  1068 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:22.004240  1068 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.004961  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:22.006363   896 leader_election.cc:343] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.008272   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.009979   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.010574   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.018769  1045 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.019898  1045 ts_tablet_manager.cc:1403] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.223s	user 0.032s	sys 0.028s
I20260709 13:12:22.020594  1057 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.025539  1057 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:22.025739   967 leader_election.cc:343] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.024878   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.027563  1004 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:22.030310   896 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.030438  1045 raft_consensus.cc:359] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.031234  1045 raft_consensus.cc:385] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.031526  1045 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.030725  1057 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.031677  1099 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:12:22.035169   967 leader_election.cc:343] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.036504  1099 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:12:22.037263   967 leader_election.cc:343] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.031854   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:22.035162  1045 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.034819  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.037259  1099 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.039922  1045 ts_tablet_manager.cc:1434] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:12:22.040355   854 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:22.042590   896 leader_election.cc:343] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.043131   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.044591  1106 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.050585   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:22.051209   853 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.054103  1106 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:22.048457   895 leader_election.cc:343] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.059324  1046 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.060067  1046 ts_tablet_manager.cc:1403] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.005s
I20260709 13:12:22.061029   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.070731  1057 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:22.072171   895 leader_election.cc:343] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.072626   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.070735   854 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:22.074494  1141 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.075450   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.076112   753 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.076861  1045 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
W20260709 13:12:22.084185   967 leader_election.cc:343] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.086618  1057 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:22.076139  1099 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.082448   751 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:22.090673  1141 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.092713  1044 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.093168  1044 ts_tablet_manager.cc:1403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.227s	user 0.016s	sys 0.001s
I20260709 13:12:22.096010  1044 raft_consensus.cc:359] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.096844  1044 raft_consensus.cc:385] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.097178  1044 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.098430  1044 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.099687   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:22.090183  1046 raft_consensus.cc:359] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.101172  1046 raft_consensus.cc:385] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.100672  1044 ts_tablet_manager.cc:1434] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:22.102021  1046 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.102666  1044 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.106712  1046 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.109246  1046 ts_tablet_manager.cc:1434] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:12:22.113076  1045 tablet_bootstrap.cc:654] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.113904   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:22.114081   896 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.117017  1158 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.117677  1158 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:22.118554  1158 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.121975  1046 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.125356  1044 tablet_bootstrap.cc:654] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.133150  1046 tablet_bootstrap.cc:654] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.142942  1044 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.143784  1044 ts_tablet_manager.cc:1403] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.004s
I20260709 13:12:22.146517  1044 raft_consensus.cc:359] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.147542  1045 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.148417  1044 raft_consensus.cc:385] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.148582  1045 ts_tablet_manager.cc:1403] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.072s	user 0.019s	sys 0.007s
I20260709 13:12:22.148844  1044 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.148197   752 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:22.149652  1044 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.151096  1045 raft_consensus.cc:359] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.151746  1045 raft_consensus.cc:385] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.152051  1045 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.153016  1044 ts_tablet_manager.cc:1434] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:12:22.154465  1044 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.152765  1045 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.161981  1045 ts_tablet_manager.cc:1434] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:12:22.162899  1045 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.166641  1044 tablet_bootstrap.cc:654] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.170910  1045 tablet_bootstrap.cc:654] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.177053  1045 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.178046  1045 ts_tablet_manager.cc:1403] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 13:12:22.179993  1046 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.180298  1044 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.180481  1046 ts_tablet_manager.cc:1403] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.007s
I20260709 13:12:22.180742  1044 ts_tablet_manager.cc:1403] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.002s
I20260709 13:12:22.180770  1045 raft_consensus.cc:359] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.181605  1045 raft_consensus.cc:385] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.181874  1045 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.182495  1045 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.183176  1046 raft_consensus.cc:359] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.183907  1046 raft_consensus.cc:385] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.184330  1046 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.184494  1045 ts_tablet_manager.cc:1434] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:12:22.185410  1045 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.185137  1046 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.191395  1045 tablet_bootstrap.cc:654] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.203243  1141 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.203948  1141 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.205811  1046 ts_tablet_manager.cc:1434] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:12:22.207595   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.208746  1141 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.209215  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.210512  1046 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
W20260709 13:12:22.210954   821 leader_election.cc:343] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:22.211910   818 leader_election.cc:343] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.212400   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.214515  1044 raft_consensus.cc:359] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.215196  1044 raft_consensus.cc:385] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.215463  1044 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.217415  1141 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.216130  1044 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.221192  1046 tablet_bootstrap.cc:654] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.230166  1044 ts_tablet_manager.cc:1434] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.000s
I20260709 13:12:22.232710  1054 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.233227  1054 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.235441  1054 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.237699  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.238318  1046 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.239207  1046 ts_tablet_manager.cc:1403] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.004s
I20260709 13:12:22.237699   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.239926   930 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:22.239967  1099 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.240643  1099 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:22.241242   821 leader_election.cc:343] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.241894  1046 raft_consensus.cc:359] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.242574  1046 raft_consensus.cc:385] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.242417   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:22.242837  1046 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.243817  1141 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:22.243451  1046 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.244185  1141 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.244544  1141 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.246102  1046 ts_tablet_manager.cc:1434] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:22.243242  1099 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.246819   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.248061  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.248399  1044 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
W20260709 13:12:22.249418   896 leader_election.cc:343] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.250052  1054 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.250579  1054 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:22.251732   895 leader_election.cc:343] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.255703   895 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.256688  1044 tablet_bootstrap.cc:654] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.256747  1057 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.259939  1046 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.260119   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.260679   930 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:22.261966   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.262874  1142 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:22.263015  1057 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.263252  1142 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.263650  1142 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.263453  1057 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.254894  1141 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.265934  1045 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.266412  1057 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.267182  1045 ts_tablet_manager.cc:1403] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.082s	user 0.030s	sys 0.038s
I20260709 13:12:22.267335   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.269179  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.269608  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.270198  1044 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
W20260709 13:12:22.270637   895 leader_election.cc:343] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:22.270475   821 leader_election.cc:343] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.270475  1045 raft_consensus.cc:359] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:22.271256   896 leader_election.cc:343] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.271482  1045 raft_consensus.cc:385] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.271842  1044 ts_tablet_manager.cc:1403] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 13:12:22.271816  1045 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.271703   896 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.272727  1057 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.272714  1045 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.274353  1045 ts_tablet_manager.cc:1434] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:22.275240  1045 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.266686  1054 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.276290  1142 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.279462   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.280097   930 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.280247   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.279817  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:22.280889   929 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:22.282030   821 leader_election.cc:343] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.282584  1141 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.286966   929 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:22.288271   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.289223  1142 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.289149  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:22.289149  1054 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.290228  1054 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:12:22.290490   821 leader_election.cc:343] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.291054  1054 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.288594   930 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:22.294312  1046 tablet_bootstrap.cc:654] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.294878   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:22.296290  1142 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.296798  1142 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:22.297643  1142 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.298769  1045 tablet_bootstrap.cc:654] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.306505   752 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.312857  1044 raft_consensus.cc:359] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.313371  1044 raft_consensus.cc:385] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.313609  1044 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.314394  1044 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.316601  1046 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.317022  1046 ts_tablet_manager.cc:1403] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.007s
I20260709 13:12:22.321095  1045 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.321563  1045 ts_tablet_manager.cc:1403] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.004s
I20260709 13:12:22.326129  1046 raft_consensus.cc:359] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.326807  1046 raft_consensus.cc:385] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.326568  1045 raft_consensus.cc:359] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.327067  1046 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.327203  1045 raft_consensus.cc:385] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.327467  1045 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.327864  1044 ts_tablet_manager.cc:1434] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:12:22.327770  1046 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.328826  1044 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.328083  1045 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.336069  1044 tablet_bootstrap.cc:654] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.338558  1158 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.339102  1158 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.340564  1045 ts_tablet_manager.cc:1434] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:12:22.341250  1158 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.341964   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.344156  1045 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.344712   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.346148   967 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.347522  1158 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.349169  1044 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.348394   753 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.349601  1044 ts_tablet_manager.cc:1403] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.000s
I20260709 13:12:22.351434  1044 raft_consensus.cc:359] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.352003  1044 raft_consensus.cc:385] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.352254  1044 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.352859  1046 ts_tablet_manager.cc:1434] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:12:22.352874  1044 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.353892  1046 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.355196  1044 ts_tablet_manager.cc:1434] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:12:22.356127  1044 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.359366  1046 tablet_bootstrap.cc:654] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.361581  1044 tablet_bootstrap.cc:654] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.370833  1046 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.370911  1044 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.371338  1046 ts_tablet_manager.cc:1403] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.001s
I20260709 13:12:22.371456  1044 ts_tablet_manager.cc:1403] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:12:22.373957  1046 raft_consensus.cc:359] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.374208  1044 raft_consensus.cc:359] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.374784  1046 raft_consensus.cc:385] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.374953  1044 raft_consensus.cc:385] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.375116  1046 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.375310  1044 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.375844  1046 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.376184  1044 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.378759  1044 ts_tablet_manager.cc:1434] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:22.379900  1044 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.382014  1046 ts_tablet_manager.cc:1434] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:22.382933  1046 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.383864  1004 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.388656  1045 tablet_bootstrap.cc:654] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.388652  1046 tablet_bootstrap.cc:654] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.400280  1046 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.401203  1046 ts_tablet_manager.cc:1403] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:12:22.403857  1046 raft_consensus.cc:359] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.404479  1046 raft_consensus.cc:385] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.404738  1046 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.408793  1044 tablet_bootstrap.cc:654] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:22.409531   969 consensus_peers.cc:597] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.410172  1158 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:22.410945  1045 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.411540  1045 ts_tablet_manager.cc:1403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.068s	user 0.006s	sys 0.011s
I20260709 13:12:22.411371  1046 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.413762  1046 ts_tablet_manager.cc:1434] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:12:22.414295  1045 raft_consensus.cc:359] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.414961  1045 raft_consensus.cc:385] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.415294  1046 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.415220  1045 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.422719  1046 tablet_bootstrap.cc:654] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.436093  1158 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.438225  1045 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.442183  1045 ts_tablet_manager.cc:1434] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:12:22.443436  1045 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.450428  1140 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.450968  1140 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.453336  1140 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.456037  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.457038   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.457669   929 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:22.461097   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.462280  1140 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:22.462347   821 leader_election.cc:343] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.462723  1140 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.463073  1140 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.467862   929 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.467850  1057 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.474419  1057 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.476537  1057 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.476850  1046 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.477326  1046 ts_tablet_manager.cc:1403] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.011s
I20260709 13:12:22.482134  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.482702   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:22.484236   895 leader_election.cc:343] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.482812  1004 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:22.485852   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.487519  1057 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:22.488008  1057 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.488406  1057 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.494093  1046 raft_consensus.cc:359] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.494823  1046 raft_consensus.cc:385] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.495112  1046 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.487017  1140 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.495922  1046 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.498311  1077 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.498783  1077 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.501312  1077 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.503199  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.504699   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.505303   930 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:22.506317   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.507099  1077 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:22.507427  1077 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.507787  1077 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.510212  1046 ts_tablet_manager.cc:1434] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
W20260709 13:12:22.514675   821 leader_election.cc:343] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.516584  1046 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.516487  1145 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:22.525501  1077 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.527753  1045 tablet_bootstrap.cc:654] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.529011   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.529625   930 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.530992  1097 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:22.532266  1044 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.532685  1044 ts_tablet_manager.cc:1403] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.153s	user 0.019s	sys 0.043s
I20260709 13:12:22.535547  1140 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.536666  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:12:22.538311   821 leader_election.cc:343] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.538179  1044 raft_consensus.cc:359] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.538892  1044 raft_consensus.cc:385] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.539168  1044 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.539119   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:22.539878   927 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.540431  1046 tablet_bootstrap.cc:654] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.542263  1044 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.523837  1057 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.544548  1044 ts_tablet_manager.cc:1434] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:12:22.545485  1044 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.545786   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:22.547358  1145 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:22.547765   895 leader_election.cc:343] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.553277  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:22.551226   930 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:22.554160  1077 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:22.555734   821 leader_election.cc:343] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.559746  1057 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.561733  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:22.561304   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:22.562505  1003 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.571555   854 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.566843  1068 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.576936  1068 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.580176  1068 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.580963   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.581736   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.583917   926 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:22.585269   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.574296   927 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:22.586525  1158 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.587121   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:22.566512  1077 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.588802  1140 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.589426  1140 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:22.590418  1140 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.572415  1003 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:22.593926  1077 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:22.596438  1077 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.598529  1045 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.625510  1045 ts_tablet_manager.cc:1403] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.182s	user 0.015s	sys 0.003s
I20260709 13:12:22.628839  1149 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.631726  1045 raft_consensus.cc:359] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.615904   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.629384  1149 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.634680  1045 raft_consensus.cc:385] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:22.622007   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:22.635838  1045 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.605499  1106 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.636493  1045 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.640708  1044 tablet_bootstrap.cc:654] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.644160  1045 ts_tablet_manager.cc:1434] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:12:22.645578  1045 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.630313  1158 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.636967  1097 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.661725  1046 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.662393  1046 ts_tablet_manager.cc:1403] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.146s	user 0.013s	sys 0.003s
W20260709 13:12:22.656392   967 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.643038  1106 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.661043  1097 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:22.672111  1097 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.692082  1046 raft_consensus.cc:359] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.692793  1046 raft_consensus.cc:385] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.693035  1046 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.694607  1046 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.695339  1149 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.697177   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.697705   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.693691  1004 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:22.700074   969 leader_election.cc:343] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:22.700770   969 leader_election.cc:343] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.718457  1099 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.719812  1099 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.721104   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.721935  1099 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:22.723392   967 leader_election.cc:343] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.723845   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.724557   851 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:22.726595   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.727128   751 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.724617  1106 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.728039   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.746448  1045 tablet_bootstrap.cc:654] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.749771  1106 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.736307  1099 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:22.762027   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.764626  1003 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:22.786842   967 leader_election.cc:343] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.794384  1046 ts_tablet_manager.cc:1434] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.132s	user 0.020s	sys 0.005s
I20260709 13:12:22.795907  1046 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.772769   751 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.760567  1099 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.794253   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.804445  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.804874  1068 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.762027  1057 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:12:22.800910  1099 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.810920  1044 tablet_bootstrap.cc:492] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.811338  1044 ts_tablet_manager.cc:1403] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.266s	user 0.007s	sys 0.016s
I20260709 13:12:22.817684   753 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.837428  1044 raft_consensus.cc:359] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.838434  1044 raft_consensus.cc:385] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.838779  1044 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.824709  1097 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.839666  1044 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.843355  1046 tablet_bootstrap.cc:654] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.846196  1044 ts_tablet_manager.cc:1434] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:12:22.846922  1044 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.850713  1149 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.842381  1097 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.851330  1149 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.854061  1149 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.858938   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.860085   929 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:22.861446   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.862090  1106 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.862594  1106 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.864288   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:22.864820  1142 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:22.865121  1149 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.874542   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.875195   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:22.877241   895 leader_election.cc:343] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.878134  1106 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.871637  1099 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.879865   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:22.880491   929 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:12:22.880908   896 leader_election.cc:343] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.881459  1099 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.865585  1097 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.883920   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.883186   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:22.888206   854 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.893543  1044 tablet_bootstrap.cc:654] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.895402  1046 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:22.895922  1046 ts_tablet_manager.cc:1403] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.100s	user 0.008s	sys 0.006s
I20260709 13:12:22.895641  1142 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.898777  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:22.900796  1149 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.903857  1045 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:22.904167   854 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:22.903925  1140 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.904422  1045 ts_tablet_manager.cc:1403] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.259s	user 0.012s	sys 0.030s
I20260709 13:12:22.900692  1046 raft_consensus.cc:359] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.905213  1046 raft_consensus.cc:385] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.905494  1046 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.905553   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:22.907462  1057 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.907562  1045 raft_consensus.cc:359] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.908049  1057 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:22.908419  1045 raft_consensus.cc:385] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.908712  1045 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.908860  1077 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:22.908944  1057 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.910013  1045 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.914122  1106 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:12:22.914155   896 leader_election.cc:343] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.914958   896 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:22.915864  1106 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:22.916323  1106 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:22.917347  1099 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:22.917480  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:22.921324  1044 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:22.919656  1045 ts_tablet_manager.cc:1434] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 13:12:22.921809  1044 ts_tablet_manager.cc:1403] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.075s	user 0.004s	sys 0.006s
I20260709 13:12:22.924188  1106 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.924525  1044 raft_consensus.cc:359] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.925148  1044 raft_consensus.cc:385] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.925429  1044 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.926014  1044 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.928979  1045 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:22.935461  1097 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.936293  1097 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:22.938967  1097 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:22.939097   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:22.906140  1046 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:22.941087   895 leader_election.cc:343] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.941360  1046 ts_tablet_manager.cc:1434] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 13:12:22.943019  1046 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:22.946563  1044 ts_tablet_manager.cc:1434] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:12:22.947875  1044 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:22.945809   752 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:22.953308  1045 tablet_bootstrap.cc:654] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.958703  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:22.961589   896 leader_election.cc:343] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:22.981060  1003 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:22.988793  1106 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:22.998844  1046 tablet_bootstrap.cc:654] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.008525   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:23.009828  1077 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:23.010957  1209 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:23.011314  1209 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:23.012126  1209 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.013233  1209 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:23.013396   821 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:23.021595  1044 tablet_bootstrap.cc:654] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.015506   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:23.026580  1142 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.005831   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44530.3us
I20260709 13:12:23.029892  1208 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.029956  1142 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.030565  1208 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.032732  1142 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.032872  1208 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.036362  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:23.036484   896 leader_election.cc:343] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.036484  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.041064  1003 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:23.037293   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:12:23.038159   821 leader_election.cc:343] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.052420  1057 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:23.050910   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:23.058495  1142 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:23.058969  1142 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.059492  1142 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.070873  1045 tablet_bootstrap.cc:492] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.070659   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.071347   930 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:23.074301  1046 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:23.075158   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.071349  1045 ts_tablet_manager.cc:1403] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.143s	user 0.035s	sys 0.029s
I20260709 13:12:23.077917   854 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:23.081769  1077 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:23.084151  1142 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.085474   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:23.086169  1142 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.087191  1106 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:23.087857  1106 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:23.088514  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:23.089164  1003 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.088755  1106 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.092365   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:23.094542  1077 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:23.075150  1046 ts_tablet_manager.cc:1403] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.001s
I20260709 13:12:23.094964  1077 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.097527  1077 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.100315  1046 raft_consensus.cc:359] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.104224  1046 raft_consensus.cc:385] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.104585  1046 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.105407  1046 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.107645  1046 ts_tablet_manager.cc:1434] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:23.102336  1003 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:23.102820   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:23.096978   818 leader_election.cc:343] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.088790  1045 raft_consensus.cc:359] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.108767  1046 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.110245  1045 raft_consensus.cc:385] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.110563  1045 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.111202  1045 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.114154  1045 ts_tablet_manager.cc:1434] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 13:12:23.115147  1045 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
W20260709 13:12:23.117276   818 leader_election.cc:343] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.118266  1046 tablet_bootstrap.cc:654] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.124774   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:23.125877  1142 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:23.126333  1142 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:23.118459   753 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:23.127352  1142 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.096485  1044 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:23.129542  1044 ts_tablet_manager.cc:1403] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.182s	user 0.009s	sys 0.011s
I20260709 13:12:23.132899  1077 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.143177  1045 tablet_bootstrap.cc:654] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.151180   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:23.151976   929 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.159545  1044 raft_consensus.cc:359] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.167168  1208 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:23.171924  1057 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.172398  1057 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.169823  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:23.174458  1057 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.175701  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:23.177418   896 leader_election.cc:343] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:23.179595   821 leader_election.cc:343] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.171929   929 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:23.195150   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:23.187954   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.170466  1077 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.200385  1077 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:23.188477  1044 raft_consensus.cc:385] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:23.200758   895 leader_election.cc:343] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.200949  1044 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.201221   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:23.201753  1044 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.202474  1057 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:23.202914  1208 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:23.192644   854 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.203671   752 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:23.205682  1077 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:23.206534  1077 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.212297  1044 ts_tablet_manager.cc:1434] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.001s
I20260709 13:12:23.213253  1044 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:23.217957  1045 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.219893  1057 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:23.228228  1045 ts_tablet_manager.cc:1403] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.113s	user 0.013s	sys 0.000s
I20260709 13:12:23.230911  1045 raft_consensus.cc:359] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.231576  1045 raft_consensus.cc:385] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.231881  1045 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.232488  1045 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.234643  1045 ts_tablet_manager.cc:1434] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:12:23.235628  1045 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.246644  1044 tablet_bootstrap.cc:654] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.253746  1045 tablet_bootstrap.cc:654] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.261179  1046 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:23.261638  1046 ts_tablet_manager.cc:1403] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.153s	user 0.018s	sys 0.014s
I20260709 13:12:23.265791  1046 raft_consensus.cc:359] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.266781  1046 raft_consensus.cc:385] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.267062  1046 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.269441  1044 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:23.270026  1044 ts_tablet_manager.cc:1403] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.057s	user 0.016s	sys 0.003s
I20260709 13:12:23.271953  1046 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.274013  1046 ts_tablet_manager.cc:1434] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:12:23.279420  1046 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.291054  1057 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:23.292284  1044 raft_consensus.cc:359] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.293035  1044 raft_consensus.cc:385] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.293310  1044 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.293965  1044 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.296036  1044 ts_tablet_manager.cc:1434] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:12:23.301309  1045 tablet_bootstrap.cc:492] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.302060  1045 ts_tablet_manager.cc:1403] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.014s
I20260709 13:12:23.303241  1044 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:23.304281  1057 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.301434   750 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:23.304731  1057 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.306128  1045 raft_consensus.cc:359] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.306900  1045 raft_consensus.cc:385] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.307199  1045 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.308070  1045 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.310670  1045 ts_tablet_manager.cc:1434] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:12:23.311344  1045 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.312628   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.313251   854 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:23.314437  1057 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.314610   895 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:23.315074  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.315562  1057 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:23.315714  1002 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:23.316054  1057 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.316587  1057 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.322362  1046 tablet_bootstrap.cc:654] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.325650  1045 tablet_bootstrap.cc:654] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.333036  1045 tablet_bootstrap.cc:492] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.333565  1045 ts_tablet_manager.cc:1403] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.004s
I20260709 13:12:23.338295  1057 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.339759  1044 tablet_bootstrap.cc:654] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.341207   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:23.341531  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:23.341836   854 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.342057  1002 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.348742  1002 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:23.348744   854 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:23.351460   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:23.351493  1057 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.352787  1057 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:23.353314  1057 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:23.354084  1057 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.357975  1046 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:23.358456  1046 ts_tablet_manager.cc:1403] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.003s
I20260709 13:12:23.359547  1045 raft_consensus.cc:359] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.361295  1045 raft_consensus.cc:385] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.362004  1045 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.361037  1046 raft_consensus.cc:359] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.363256  1046 raft_consensus.cc:385] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.363586  1046 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.364446  1046 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.365263  1044 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:23.365687  1044 ts_tablet_manager.cc:1403] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.003s
I20260709 13:12:23.366541  1046 ts_tablet_manager.cc:1434] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:12:23.367545  1046 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.363059  1045 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.369298  1045 ts_tablet_manager.cc:1434] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.003s
I20260709 13:12:23.370306  1045 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.374066  1046 tablet_bootstrap.cc:654] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.375780  1045 tablet_bootstrap.cc:654] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.368012  1044 raft_consensus.cc:359] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.388303  1044 raft_consensus.cc:385] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.388679  1044 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.388659   750 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:23.389937  1044 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.414335  1044 ts_tablet_manager.cc:1434] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.004s
I20260709 13:12:23.417924  1046 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:23.418452  1046 ts_tablet_manager.cc:1403] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.000s
I20260709 13:12:23.433080  1046 raft_consensus.cc:359] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.433801  1046 raft_consensus.cc:385] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.434106  1046 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.436116  1044 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:23.436210  1046 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.437465  1045 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.437950  1045 ts_tablet_manager.cc:1403] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.068s	user 0.029s	sys 0.033s
I20260709 13:12:23.441844  1045 raft_consensus.cc:359] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.442956  1044 tablet_bootstrap.cc:654] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.443810  1045 raft_consensus.cc:385] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.444129  1045 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.443284  1046 ts_tablet_manager.cc:1434] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:12:23.444775  1045 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.447081  1046 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.451031  1003 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.459589  1045 ts_tablet_manager.cc:1434] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 13:12:23.460793  1045 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.464471  1046 tablet_bootstrap.cc:654] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.466998  1045 tablet_bootstrap.cc:654] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.478154  1044 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:23.487916  1044 ts_tablet_manager.cc:1403] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.007s
I20260709 13:12:23.490470  1044 raft_consensus.cc:359] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.491112  1044 raft_consensus.cc:385] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.491390  1044 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.492161  1044 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.494297  1044 ts_tablet_manager.cc:1434] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:12:23.495334  1044 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:23.497927  1046 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
W20260709 13:12:23.499723   896 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:23.500331  1045 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.501965  1045 ts_tablet_manager.cc:1403] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.006s
I20260709 13:12:23.501006  1046 ts_tablet_manager.cc:1403] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.010s
I20260709 13:12:23.505996  1045 raft_consensus.cc:359] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.506687  1045 raft_consensus.cc:385] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.506958  1045 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.507572  1045 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.515900  1044 tablet_bootstrap.cc:654] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.517737  1045 ts_tablet_manager.cc:1434] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.004s
I20260709 13:12:23.518886  1045 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.519277  1046 raft_consensus.cc:359] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.519946  1046 raft_consensus.cc:385] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.520210  1046 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.520839  1046 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.524703  1046 ts_tablet_manager.cc:1434] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:23.524811  1045 tablet_bootstrap.cc:654] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.528040  1106 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.528534  1106 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.530611  1106 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:23.532025  1046 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.532990   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.534248   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.534899   930 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:23.540536  1044 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:23.541061  1044 ts_tablet_manager.cc:1403] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.000s
I20260709 13:12:23.543767  1044 raft_consensus.cc:359] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.544458  1044 raft_consensus.cc:385] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.544720  1044 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.551236  1044 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.540586   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:12:23.554970   969 leader_election.cc:343] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.558182  1145 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:23.558586  1145 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.558938  1145 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.565047  1149 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.565562  1149 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.573166   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.573915   855 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:23.571697  1149 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:23.576103   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:23.576481   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.575839  1057 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:23.577046  1149 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:23.577117  1045 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:23.577473  1149 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.588292  1044 ts_tablet_manager.cc:1434] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.003s
I20260709 13:12:23.577986  1045 ts_tablet_manager.cc:1403] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.008s
I20260709 13:12:23.604981  1044 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:23.577838  1149 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.584523  1046 tablet_bootstrap.cc:654] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.606699  1145 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.580721   929 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:23.615401  1045 raft_consensus.cc:359] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.618446  1045 raft_consensus.cc:385] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.618839  1045 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.619836  1045 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.576527  1077 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 13:12:23.577247   930 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:23.626215  1057 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.626744  1057 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.628315  1149 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.629132   854 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:23.628821  1057 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.632663  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.634301  1002 raft_consensus.cc:2393] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:23.634562   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.635792  1145 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:23.637647   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:23.638129   821 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:23.638279   851 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.638758  1142 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:23.639022   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:23.642638  1196 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.647776  1045 ts_tablet_manager.cc:1434] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 13:12:23.649130  1045 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:23.650777  1145 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.655035  1145 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.657934  1046 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:23.658558  1046 ts_tablet_manager.cc:1403] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.127s	user 0.025s	sys 0.018s
I20260709 13:12:23.660216   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:23.661065   930 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.663317  1046 raft_consensus.cc:359] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.664294  1046 raft_consensus.cc:385] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:23.664641  1046 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.644631  1106 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:12:23.674378   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:23.675086   853 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.678987  1149 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:23.691195  1099 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.693024  1099 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.696166  1057 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.696754  1057 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.697901  1140 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.725397  1046 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.730649  1046 ts_tablet_manager.cc:1434] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.002s
I20260709 13:12:23.731484  1046 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:23.742166   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:23.742818   928 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.726539  1140 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.764315  1209 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.780887   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:23.783378   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:23.776628  1209 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.785251  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.774784  1099 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.792068  1003 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:23.793439   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:12:23.794364   895 leader_election.cc:343] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.793530  1097 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.802409  1044 tablet_bootstrap.cc:654] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.796186   927 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:23.810338  1045 tablet_bootstrap.cc:654] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.797631  1097 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:23.816682  1237 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:12:23.819146  1238 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:12:23.820976  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.827555  1149 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.827984  1209 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.809923  1145 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:23.828285  1149 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.814890  1097 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.812530   930 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:23.838245  1057 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.827088   928 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:23.854347  1140 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:23.863339   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.862056  1240 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.893334   853 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:12:23.893716   969 leader_election.cc:343] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:23.894338   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.904175   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.904628   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:23.863571  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.905298   926 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:23.943315   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:23.932303  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:23.917079  1004 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:23.916512  1240 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:23.954088   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:12:24.021401   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79215.9us
I20260709 13:12:24.021644  1242 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.022174  1242 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:24.022661  1196 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.023046  1242 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.023797   929 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:24.024840  1001 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
W20260709 13:12:23.916419   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 87548.6us
I20260709 13:12:24.045933  1046 tablet_bootstrap.cc:654] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.026530  1149 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:24.031884  1240 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.037830  1244 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.053442  1244 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.053841  1244 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.064477  1099 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:24.065112  1099 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:24.073516   896 leader_election.cc:343] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.074118   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:24.074818   852 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:24.074295   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:24.087522  1196 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:24.090458   818 leader_election.cc:343] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.090665  1196 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.091042  1196 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.091832   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:24.086087   851 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:24.095834  1149 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.104718   852 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.122179   818 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:24.124143  1208 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.124541  1208 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.107076  1099 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.099098   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:24.121805  1149 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.124812  1140 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:12:24.134979  1149 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.124857  1208 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.095278   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:24.144457   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:24.145465  1045 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.146209  1045 ts_tablet_manager.cc:1403] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.497s	user 0.026s	sys 0.029s
I20260709 13:12:24.150517  1142 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.151196  1142 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.151651  1142 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.152593   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:24.156153  1239 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.156908  1239 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:24.155035   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:24.130867  1244 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.173818  1142 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:24.160641   895 leader_election.cc:343] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:24.160343   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70315.2us
I20260709 13:12:24.179350   928 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:24.157991  1239 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.181604  1044 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.181983  1044 ts_tablet_manager.cc:1403] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.577s	user 0.016s	sys 0.030s
W20260709 13:12:24.183219   895 leader_election.cc:343] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.184566   895 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.185535  1099 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.187685  1142 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.189661  1045 raft_consensus.cc:359] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.190343  1045 raft_consensus.cc:385] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.190598  1045 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.191203  1045 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.164158  1149 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.149772  1196 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.194833  1145 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.181223  1244 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.197142  1196 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:24.200084  1254 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:12:24.206003   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:24.202303   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:24.213874   851 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:24.218726  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.220533  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.245394  1045 ts_tablet_manager.cc:1434] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.099s	user 0.004s	sys 0.002s
I20260709 13:12:24.246590  1045 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:24.261847  1046 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.262410  1046 ts_tablet_manager.cc:1403] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.531s	user 0.040s	sys 0.034s
I20260709 13:12:24.267683   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:24.272004  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.276733  1196 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.278954  1208 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.281255  1046 raft_consensus.cc:359] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.281937  1046 raft_consensus.cc:385] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.282197  1046 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.282228  1044 raft_consensus.cc:359] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.283002  1044 raft_consensus.cc:385] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.283290  1044 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.284013  1044 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.284970  1046 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.286096  1044 ts_tablet_manager.cc:1434] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.104s	user 0.000s	sys 0.006s
I20260709 13:12:24.286932  1046 ts_tablet_manager.cc:1434] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:12:24.288335  1046 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:24.268272  1149 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:24.287717  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:12:24.291544   818 leader_election.cc:343] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.296150  1002 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.298919   752 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:24.301329   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:24.304828   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:24.305461   999 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.308060  1142 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.308214   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:24.308564  1142 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.315222  1142 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.327205  1044 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.302675  1208 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.339810  1196 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.340480  1196 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.342442  1196 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.330066   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:24.338974  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:24.349857  1004 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:12:24.356386  1259 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:12:24.359547  1244 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.359781   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.360698  1196 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:24.363736  1002 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:24.363880  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:24.361698   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:24.365031  1001 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.364231   999 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:24.365116   925 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.368639   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:24.368851   821 leader_election.cc:343] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.367794  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.369282   927 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:24.371685   821 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:24.372467  1142 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.373615  1046 tablet_bootstrap.cc:654] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.368458  1045 tablet_bootstrap.cc:654] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:24.377036   895 leader_election.cc:343] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.378328   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:24.382946   895 leader_election.cc:343] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.382417  1208 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:12:24.383798   821 leader_election.cc:343] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.392686  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.385367  1044 tablet_bootstrap.cc:654] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.387300   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:24.395509  1077 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.395977  1077 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.396354  1077 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.400115   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:24.401774   967 leader_election.cc:343] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.403784   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:24.404592  1149 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:24.406821   821 leader_election.cc:343] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.416155   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:24.420864  1045 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.421319  1045 ts_tablet_manager.cc:1403] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.175s	user 0.019s	sys 0.005s
I20260709 13:12:24.408591   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.428947  1196 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.429600  1196 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:24.429935  1046 tablet_bootstrap.cc:492] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.430444  1046 ts_tablet_manager.cc:1403] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.142s	user 0.009s	sys 0.005s
I20260709 13:12:24.430433  1196 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.433683  1046 raft_consensus.cc:359] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.434340  1046 raft_consensus.cc:385] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.434623  1046 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
W20260709 13:12:24.435917   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47544.1us
I20260709 13:12:24.436897   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.435506  1046 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.437911  1244 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:12:24.438663   896 leader_election.cc:343] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.442565  1046 ts_tablet_manager.cc:1434] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:12:24.443034  1244 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:24.445155  1046 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:24.445008  1244 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:24.454525   967 leader_election.cc:343] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.456518   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:24.455269   925 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:24.457705  1149 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.439886  1077 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.472191   855 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:24.477197  1077 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.479504  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:12:24.485435   821 leader_election.cc:343] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.490662  1045 raft_consensus.cc:359] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.493432  1045 raft_consensus.cc:385] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.493825  1045 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.495774   818 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:24.497406  1208 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.497864  1208 raft_consensus.cc:697] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:24.498739  1208 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.503379  1045 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.505709   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:24.505919  1045 ts_tablet_manager.cc:1434] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.084s	user 0.010s	sys 0.000s
I20260709 13:12:24.506330   928 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.506987  1045 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:24.522213   928 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:24.526714  1044 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.527172   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.527210  1044 ts_tablet_manager.cc:1403] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.200s	user 0.012s	sys 0.003s
I20260709 13:12:24.531901  1046 tablet_bootstrap.cc:654] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.532172   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:24.534230  1142 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.534694  1142 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:24.538910  1044 raft_consensus.cc:359] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.539588   753 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.539587  1044 raft_consensus.cc:385] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.541049  1044 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.542505  1044 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.545693  1044 ts_tablet_manager.cc:1434] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 13:12:24.535511  1142 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.547590  1253 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.552675  1044 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.558655  1045 tablet_bootstrap.cc:654] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.559969  1196 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.560456  1196 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.563222  1196 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.567996  1044 tablet_bootstrap.cc:654] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.568039  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.568902  1001 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:24.564859   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:24.569962   855 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:24.571983  1046 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.572438  1046 ts_tablet_manager.cc:1403] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.128s	user 0.008s	sys 0.010s
I20260709 13:12:24.573805   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.574823  1196 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.573167   750 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.578409  1149 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.580204   750 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.574924  1046 raft_consensus.cc:359] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.586757  1046 raft_consensus.cc:385] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.587543  1046 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.588771  1046 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.592336  1149 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.593246  1046 ts_tablet_manager.cc:1434] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:24.594115  1046 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:24.598973   753 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.606873  1239 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:24.616220  1046 tablet_bootstrap.cc:654] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.622496  1196 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.636649  1045 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.637508  1045 ts_tablet_manager.cc:1403] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.131s	user 0.046s	sys 0.009s
I20260709 13:12:24.637559  1044 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.638674  1044 ts_tablet_manager.cc:1403] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.086s	user 0.006s	sys 0.007s
I20260709 13:12:24.641153  1045 raft_consensus.cc:359] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.641460  1044 raft_consensus.cc:359] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.644548  1044 raft_consensus.cc:385] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.644937  1044 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.644585  1045 raft_consensus.cc:385] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.645465  1045 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.645793  1044 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.646508  1045 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.648118  1044 ts_tablet_manager.cc:1434] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:12:24.649067  1044 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.650691  1046 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.651170  1046 ts_tablet_manager.cc:1403] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.001s
I20260709 13:12:24.654748  1044 tablet_bootstrap.cc:654] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.658057  1046 raft_consensus.cc:359] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.660038  1046 raft_consensus.cc:385] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.660336  1046 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.660971  1046 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.661528  1045 ts_tablet_manager.cc:1434] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 13:12:24.663180  1046 ts_tablet_manager.cc:1434] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.005s
I20260709 13:12:24.668191  1046 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:24.662586  1045 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:24.677316  1046 tablet_bootstrap.cc:654] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.689141  1045 tablet_bootstrap.cc:654] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.697396  1145 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:24.754611  1046 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.755076  1046 ts_tablet_manager.cc:1403] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.087s	user 0.012s	sys 0.023s
I20260709 13:12:24.755256  1253 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.756860  1253 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.759253  1253 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.761060  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:24.763937   896 leader_election.cc:343] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.767343  1044 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.767151   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:24.768007  1044 ts_tablet_manager.cc:1403] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.119s	user 0.021s	sys 0.021s
W20260709 13:12:24.769140   895 leader_election.cc:343] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.769670   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:24.770690  1253 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.771246  1044 raft_consensus.cc:359] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.771780  1045 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.772084  1044 raft_consensus.cc:385] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.772267  1045 ts_tablet_manager.cc:1403] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.043s
I20260709 13:12:24.772413  1044 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.773231  1044 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.774864  1142 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.775381  1142 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.777624  1142 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.777622  1045 raft_consensus.cc:359] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.779163  1045 raft_consensus.cc:385] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.779449  1045 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.780504  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:24.780551   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:24.781351   928 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:24.781476  1045 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:24.783423   821 leader_election.cc:343] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.785465  1044 ts_tablet_manager.cc:1434] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.004s
I20260709 13:12:24.787123   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:24.788751  1044 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.789149  1208 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:24.789548  1208 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.789885  1208 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.796466  1045 ts_tablet_manager.cc:1434] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:12:24.798077  1045 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:24.801568  1208 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.804311  1045 tablet_bootstrap.cc:654] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.804445   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:24.805054   928 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.804299  1044 tablet_bootstrap.cc:654] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.816169   928 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:24.822193  1046 raft_consensus.cc:359] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.822942  1046 raft_consensus.cc:385] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.823218  1046 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.834234  1044 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.835949  1044 ts_tablet_manager.cc:1403] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.008s
W20260709 13:12:24.840762   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:24.840652   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:24.839401  1044 raft_consensus.cc:359] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.842803  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:24.842907  1142 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:24.843241  1044 raft_consensus.cc:385] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.843627  1044 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.843963  1142 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:12:24.844369   821 leader_election.cc:343] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.844292  1044 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.844957  1142 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.846455  1044 ts_tablet_manager.cc:1434] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:12:24.863507  1044 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.846762  1208 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:24.871754  1044 tablet_bootstrap.cc:654] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.847227  1046 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.883652  1046 ts_tablet_manager.cc:1434] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.128s	user 0.003s	sys 0.007s
I20260709 13:12:24.885038  1145 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.885351  1253 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.885944  1044 tablet_bootstrap.cc:492] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:24.885545  1145 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.886364  1044 ts_tablet_manager.cc:1403] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.008s
I20260709 13:12:24.888423   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:24.888932  1044 raft_consensus.cc:359] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.890105  1044 raft_consensus.cc:385] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.890420  1145 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:24.890468  1044 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.891526   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:24.891515  1044 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.892870   928 raft_consensus.cc:2393] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:24.893471  1045 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.893931  1045 ts_tablet_manager.cc:1403] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.096s	user 0.020s	sys 0.007s
I20260709 13:12:24.894241  1044 ts_tablet_manager.cc:1434] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:24.894096   967 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:24.895179  1044 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:24.895555  1149 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.896798  1045 raft_consensus.cc:359] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.897507  1045 raft_consensus.cc:385] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.897784  1045 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.898602  1045 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.900767  1045 ts_tablet_manager.cc:1434] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:24.901480  1045 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:24.902670  1046 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:24.904917   753 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.909873  1149 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.912397  1044 tablet_bootstrap.cc:654] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.920548   855 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:24.931072  1097 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:24.919783  1045 tablet_bootstrap.cc:654] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.938424  1046 tablet_bootstrap.cc:654] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.948921  1253 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:24.956884  1045 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:24.983270  1045 ts_tablet_manager.cc:1403] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.003s
I20260709 13:12:24.987474  1045 raft_consensus.cc:359] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:24.988965  1045 raft_consensus.cc:385] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.989346  1045 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.990232  1045 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:24.974500   821 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:24.961300  1046 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:24.998582  1046 ts_tablet_manager.cc:1403] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.000s
I20260709 13:12:25.001935  1046 raft_consensus.cc:359] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.003650  1046 raft_consensus.cc:385] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.004000  1046 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.005118  1046 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.007359  1045 ts_tablet_manager.cc:1434] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:12:25.010612  1045 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:25.009471  1046 ts_tablet_manager.cc:1434] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:25.008333   855 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.024497  1046 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:25.018168  1241 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.032270  1000 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.032274  1241 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.036361   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.037019   852 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:25.037276  1241 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:25.038419  1046 tablet_bootstrap.cc:654] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.040132  1045 tablet_bootstrap.cc:654] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.041517   969 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:25.042230  1208 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:25.043467  1149 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.043885  1149 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.044204  1149 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.044432  1097 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.044902  1097 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.052256  1208 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:25.045349   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.085608  1045 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:25.086192  1045 ts_tablet_manager.cc:1403] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.076s	user 0.007s	sys 0.009s
I20260709 13:12:25.092483  1097 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.080475   928 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:25.125967  1045 raft_consensus.cc:359] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.126652  1045 raft_consensus.cc:385] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.126911  1045 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.128954  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.129573  1004 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:25.130381  1149 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.132117   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:25.127480  1045 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.132967  1099 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.133368  1099 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.133710  1099 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.135099  1240 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:25.136366   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:25.136974   855 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.149241  1099 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.151239  1097 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.151480   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:25.152096   928 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.151785  1097 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.153179  1099 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.154039  1097 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.156528  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:25.156994  1004 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.161062  1045 ts_tablet_manager.cc:1434] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.075s	user 0.012s	sys 0.000s
I20260709 13:12:25.163034  1045 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:25.136005  1149 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:25.179774   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.190267   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:25.217149  1044 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:25.217811  1044 ts_tablet_manager.cc:1403] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.323s	user 0.046s	sys 0.097s
I20260709 13:12:25.257993  1004 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:25.267740   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.191063  1303 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.070s
I20260709 13:12:25.271914   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:25.279836  1044 raft_consensus.cc:359] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.280571  1044 raft_consensus.cc:385] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.280830  1044 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.280934  1303 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.281435  1303 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:25.281632  1044 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.282260  1303 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.270562  1149 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.289290  1044 ts_tablet_manager.cc:1434] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.001s
W20260709 13:12:25.266628   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 67847.2us
I20260709 13:12:25.292075  1044 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
W20260709 13:12:25.292285  1304 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:12:25.270071   855 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:25.301589  1046 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:25.302330  1046 ts_tablet_manager.cc:1403] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.278s	user 0.016s	sys 0.017s
W20260709 13:12:25.300243   896 leader_election.cc:343] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.295151  1149 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.302963  1240 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.305914  1149 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:25.306747  1240 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.316113   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.316846   850 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:25.317231  1142 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.317898  1142 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.318467   928 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:25.351318  1240 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.357024  1142 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.361759   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.346829   969 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:25.363056  1149 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.363720  1149 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:25.364178   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:25.364943  1145 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.369494  1046 raft_consensus.cc:359] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.370366  1046 raft_consensus.cc:385] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.370728  1046 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.369025  1145 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.372725  1145 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.373135   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.375339  1044 tablet_bootstrap.cc:654] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.364500  1149 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.378540  1046 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.369052   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:25.387739   895 leader_election.cc:343] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.388227   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:25.389016  1303 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.396426  1046 ts_tablet_manager.cc:1434] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.094s	user 0.010s	sys 0.021s
I20260709 13:12:25.390591   850 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.416747  1046 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:25.417189  1045 tablet_bootstrap.cc:654] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:25.428071   967 leader_election.cc:343] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.429847   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.435642  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.436606   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.426358  1145 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.444370  1044 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:25.445233  1145 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:25.488324   821 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:25.493307   850 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:25.505431   750 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:25.512143   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:25.520522   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 87370.7us
I20260709 13:12:25.522946  1001 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:25.541400  1044 ts_tablet_manager.cc:1403] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.249s	user 0.023s	sys 0.014s
I20260709 13:12:25.566497  1303 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.571867  1307 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.572816  1307 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:25.574085   818 leader_election.cc:343] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.582671  1307 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.581903   752 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.567442  1303 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.589300   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:25.590299  1303 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.590379  1046 tablet_bootstrap.cc:654] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.590474   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:25.591320   850 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.581557  1044 raft_consensus.cc:359] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.594424  1044 raft_consensus.cc:385] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.593973  1320 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:25.594832  1044 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.597154  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.599921  1044 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.594940   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:25.602269   896 leader_election.cc:343] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.601662  1045 tablet_bootstrap.cc:492] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
W20260709 13:12:25.605051   896 leader_election.cc:343] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.608125  1045 ts_tablet_manager.cc:1403] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.445s	user 0.014s	sys 0.015s
I20260709 13:12:25.608337   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:25.615267  1044 ts_tablet_manager.cc:1434] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.004s
I20260709 13:12:25.616155   818 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:25.616885  1045 raft_consensus.cc:359] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.617835  1045 raft_consensus.cc:385] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.617919  1321 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:25.618201  1045 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
W20260709 13:12:25.620415   967 leader_election.cc:343] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.623246  1044 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:25.625247  1045 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.629104   850 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:25.629856  1322 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.642088  1045 ts_tablet_manager.cc:1434] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.003s
I20260709 13:12:25.643853  1045 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:25.661293  1045 tablet_bootstrap.cc:654] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.669654   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.670071  1318 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.670585  1318 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.674086  1318 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:25.684963  1322 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.685552  1322 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.688764  1322 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.676083   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.693456  1046 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:25.699203  1046 ts_tablet_manager.cc:1403] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.283s	user 0.011s	sys 0.012s
I20260709 13:12:25.702248  1046 raft_consensus.cc:359] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.702476   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.702965  1046 raft_consensus.cc:385] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.703271  1046 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.706601   999 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:25.706950  1046 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.709231  1046 ts_tablet_manager.cc:1434] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:25.710517  1046 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:25.710325   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:25.712098   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.712893   929 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:25.714494   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.712891  1045 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:25.733461  1318 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.733783   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.734484   852 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:25.736356  1318 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:25.737816  1318 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.739934  1044 tablet_bootstrap.cc:654] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.743775  1045 ts_tablet_manager.cc:1403] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.100s	user 0.008s	sys 0.010s
I20260709 13:12:25.771656   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:25.771014   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:25.764628   929 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:25.775916  1145 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.776049  1142 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.776551  1142 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.777081  1142 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.776618  1145 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.779042  1145 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:25.783145   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 72975.7us
I20260709 13:12:25.785842  1241 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.786309  1241 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.786624  1241 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.792613   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.797991  1303 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:12:25.812497  1044 tablet_bootstrap.cc:492] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:25.813076  1044 ts_tablet_manager.cc:1403] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.190s	user 0.009s	sys 0.020s
I20260709 13:12:25.818018  1045 raft_consensus.cc:359] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.818817  1045 raft_consensus.cc:385] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.819135  1045 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
W20260709 13:12:25.819830   895 leader_election.cc:343] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.819898  1045 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.820443   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:25.821316  1303 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.821637  1045 ts_tablet_manager.cc:1434] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.077s	user 0.003s	sys 0.003s
I20260709 13:12:25.822942  1045 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:25.823985  1241 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:25.827122   969 leader_election.cc:343] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:25.828290   895 leader_election.cc:343] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.830540   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:25.836185  1307 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.844235  1044 raft_consensus.cc:359] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.845613  1044 raft_consensus.cc:385] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.845898  1044 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.847373  1044 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.849066  1142 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.846760  1307 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:12:25.851229  1142 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.852807   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:25.853008   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:25.853446   930 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.853760   852 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:25.851521  1046 tablet_bootstrap.cc:654] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.856339  1307 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.859997  1307 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.860569  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:25.862469  1241 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:25.864501   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:25.865252  1000 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:25.865298  1313 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:25.867827   930 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:25.868500  1307 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:25.869474   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:12:25.874704   967 leader_election.cc:343] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.875854  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:25.876150  1077 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.876626  1077 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:12:25.877461   896 leader_election.cc:343] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.877502  1077 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:25.878633   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:25.879503   929 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:25.883203  1045 tablet_bootstrap.cc:654] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.875195   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:25.885960   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:25.887399  1313 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.888877   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:25.890867  1241 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.910038  1044 ts_tablet_manager.cc:1434] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.097s	user 0.009s	sys 0.000s
I20260709 13:12:25.970886  1044 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
W20260709 13:12:26.066838   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:26.067744  1046 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.068421  1046 ts_tablet_manager.cc:1403] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.358s	user 0.017s	sys 0.003s
W20260709 13:12:26.076646   895 leader_election.cc:343] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.077275   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.078858  1046 raft_consensus.cc:359] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.080826  1046 raft_consensus.cc:385] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.081267  1046 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.082087  1046 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.080572  1303 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.073459  1313 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:12:26.130018  1046 ts_tablet_manager.cc:1434] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.001s
I20260709 13:12:26.137790  1337 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.138340  1337 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.140579  1337 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:26.142115   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.142019   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.145364   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.146207  1317 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.156322  1046 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:26.176116  1044 tablet_bootstrap.cc:654] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.183117  1045 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.183583  1045 ts_tablet_manager.cc:1403] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.361s	user 0.036s	sys 0.005s
I20260709 13:12:26.182890   750 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.187104  1000 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.195996  1045 raft_consensus.cc:359] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.196635  1045 raft_consensus.cc:385] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.196867  1045 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.197439  1045 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.199417  1045 ts_tablet_manager.cc:1434] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.003s
I20260709 13:12:26.201334  1045 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.208722   751 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.221850  1046 tablet_bootstrap.cc:654] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.231578  1044 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.232043  1044 ts_tablet_manager.cc:1403] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.262s	user 0.018s	sys 0.000s
I20260709 13:12:26.234297  1044 raft_consensus.cc:359] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.234957  1044 raft_consensus.cc:385] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.236129  1045 tablet_bootstrap.cc:654] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:26.240588   967 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:26.243803  1044 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.244828  1044 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.242774  1046 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.246367  1046 ts_tablet_manager.cc:1403] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.001s
I20260709 13:12:26.247162  1044 ts_tablet_manager.cc:1434] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:12:26.248238  1044 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.249061  1046 raft_consensus.cc:359] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.249737  1046 raft_consensus.cc:385] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.250010  1046 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.250772  1046 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.255214  1044 tablet_bootstrap.cc:654] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.258496  1046 ts_tablet_manager.cc:1434] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:12:26.260005  1046 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:26.274488  1045 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.275007  1045 ts_tablet_manager.cc:1403] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.002s
I20260709 13:12:26.277587  1044 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.278120  1044 ts_tablet_manager.cc:1403] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.012s
I20260709 13:12:26.278120  1045 raft_consensus.cc:359] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.279844  1046 tablet_bootstrap.cc:654] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.279793  1045 raft_consensus.cc:385] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.280230  1045 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.281826  1097 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.282150  1044 raft_consensus.cc:359] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.282357  1097 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.283731  1045 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.282881  1044 raft_consensus.cc:385] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.285830   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.286621  1044 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.288450  1240 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.288934  1240 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.290009  1045 ts_tablet_manager.cc:1434] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 13:12:26.291574  1045 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.291672   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:26.293179   818 leader_election.cc:343] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.286087  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.287079   849 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:26.288254  1044 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.296851  1044 ts_tablet_manager.cc:1434] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:12:26.297765  1044 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.299676  1240 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.304517  1309 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.305004  1309 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.307921   895 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.309090  1307 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.310108  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.310822  1003 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:26.307369  1309 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.312089   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:26.312541   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.285073  1097 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:26.315066   895 leader_election.cc:343] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.316107  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.316748  1000 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.317994   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.319001  1322 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.319371  1322 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.319716  1322 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.320363  1044 tablet_bootstrap.cc:654] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.322430  1045 tablet_bootstrap.cc:654] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.314378  1097 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.329375  1097 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.329744  1097 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.333246  1241 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.337281  1097 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.340015   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:26.340227  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.342552  1000 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.341185  1097 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.340934  1044 tablet_bootstrap.cc:492] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
W20260709 13:12:26.343220   895 leader_election.cc:343] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.345222  1044 ts_tablet_manager.cc:1403] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.001s
I20260709 13:12:26.345459  1307 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.347571  1322 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.347956  1044 raft_consensus.cc:359] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.348577  1044 raft_consensus.cc:385] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.348861  1044 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.349715  1044 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.350369   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.351364  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.351857  1044 ts_tablet_manager.cc:1434] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:26.351946  1003 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:26.351953   818 leader_election.cc:343] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.352852  1044 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.357996  1000 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:26.359454  1044 tablet_bootstrap.cc:654] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.366966   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.368170  1307 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.359597  1003 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.370270   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.371230  1240 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.371840  1240 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:26.368927  1307 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:26.372937  1240 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.363044  1045 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.380301  1045 ts_tablet_manager.cc:1403] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.089s	user 0.005s	sys 0.007s
I20260709 13:12:26.380241  1046 tablet_bootstrap.cc:492] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.384050  1046 ts_tablet_manager.cc:1403] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.003s
I20260709 13:12:26.386214  1322 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.374172  1307 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.387817  1045 raft_consensus.cc:359] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.388800  1045 raft_consensus.cc:385] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.389235  1045 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.390059  1045 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.394043  1044 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.394179  1045 ts_tablet_manager.cc:1434] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:12:26.394534  1044 ts_tablet_manager.cc:1403] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.014s
I20260709 13:12:26.387316  1046 raft_consensus.cc:359] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.395442  1045 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.395511  1046 raft_consensus.cc:385] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.396239  1046 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.396871  1046 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.397219  1044 raft_consensus.cc:359] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.397965  1044 raft_consensus.cc:385] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.398231  1044 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.398948  1046 ts_tablet_manager.cc:1434] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:12:26.400070  1046 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:26.403723  1045 tablet_bootstrap.cc:654] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.398994  1044 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.412205   752 catalog_manager.cc:5697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.420680  1046 tablet_bootstrap.cc:654] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.428360  1044 ts_tablet_manager.cc:1434] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:12:26.428972  1045 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.429430  1045 ts_tablet_manager.cc:1403] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.006s
I20260709 13:12:26.432099  1045 raft_consensus.cc:359] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.432750  1045 raft_consensus.cc:385] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.433017  1045 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.433666  1045 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.435223  1044 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.446447  1044 tablet_bootstrap.cc:654] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.450858  1045 ts_tablet_manager.cc:1434] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:26.452297  1142 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.452747  1241 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.452883  1142 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.453265  1241 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.455118  1142 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.456499   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.457161   849 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:26.457307   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.458508   927 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.458796   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:26.459656  1318 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:26.460016  1318 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.460043   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:26.460484  1318 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.461537  1322 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:26.461889  1322 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.462168  1322 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.465482   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.466256   927 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:26.469501  1241 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:26.471546  1318 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.469506   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.473373  1322 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.474738   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:26.486346   849 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:26.476601  1046 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.487363  1046 ts_tablet_manager.cc:1403] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.001s
I20260709 13:12:26.477350  1318 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:26.479257   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.487771  1322 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.487933   927 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.489082  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.489866  1003 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:26.490763  1046 raft_consensus.cc:359] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.491430  1046 raft_consensus.cc:385] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.491581  1044 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.491746  1046 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.492142  1044 ts_tablet_manager.cc:1403] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.009s
I20260709 13:12:26.492650  1046 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.492965   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.494467   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.494778  1044 raft_consensus.cc:359] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.495504  1044 raft_consensus.cc:385] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.495577  1322 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:12:26.495827  1044 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.496528  1044 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.494750  1046 ts_tablet_manager.cc:1434] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:12:26.497910  1046 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:26.498878   927 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:26.485087  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.500029  1000 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:26.502811  1044 ts_tablet_manager.cc:1434] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:26.502851   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.503821  1044 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.504136  1241 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:26.504642  1241 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:26.505477  1241 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.510473  1322 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.510979  1322 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.512629  1044 tablet_bootstrap.cc:654] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.513007  1322 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.513688   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.514374   927 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.514925  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.515547  1003 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.516956   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:26.518196  1142 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.518576  1142 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.518957  1142 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.521044  1240 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.525337  1142 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.528183  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.528784  1000 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.530663   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.531479   928 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.537537  1000 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.539774  1309 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.540508  1309 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.543985   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:26.545409   895 leader_election.cc:343] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.545637  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.546542  1003 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:26.547539  1045 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.547684   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:26.548144   896 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.549207  1097 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.549332  1240 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.549714  1097 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.550206  1097 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.553330  1240 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:26.554574  1045 tablet_bootstrap.cc:654] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.555154  1240 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.557986  1142 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.562731  1046 tablet_bootstrap.cc:654] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.573801  1309 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.576920  1321 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.577777   928 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.580879  1097 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.596482  1097 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.594671   750 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 2, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.600175   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:26.601403  1045 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.601980  1045 ts_tablet_manager.cc:1403] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.055s	user 0.032s	sys 0.000s
I20260709 13:12:26.604568  1045 raft_consensus.cc:359] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.605197  1045 raft_consensus.cc:385] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.605463  1045 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.605701  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:12:26.606168   895 leader_election.cc:343] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.606045  1045 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.606410  1003 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.610952  1045 ts_tablet_manager.cc:1434] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:12:26.614645  1142 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.615293  1142 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.617144  1142 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.618196  1241 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.618813  1241 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.619129   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.629609  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.630440  1002 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:26.635666  1322 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.636168  1322 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.637308  1302 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.637763  1302 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.634672  1003 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:26.643746   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.645449   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.646410  1142 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.644304   929 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.651885   896 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.653155  1142 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.678023  1307 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.678551  1045 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.638245  1322 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.681211   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.679528  1044 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.681998   929 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:26.682399  1044 ts_tablet_manager.cc:1403] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.179s	user 0.050s	sys 0.042s
I20260709 13:12:26.679849  1046 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.683434  1046 ts_tablet_manager.cc:1403] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.186s	user 0.011s	sys 0.015s
I20260709 13:12:26.681370  1302 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.683398   967 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:26.651651  1321 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.679889  1241 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:26.690671  1321 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.708750  1321 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.709733  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.709553  1046 raft_consensus.cc:359] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.710279  1046 raft_consensus.cc:385] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.710311  1004 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.710572  1046 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.711483  1046 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.712131   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:26.690510  1317 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.713477  1321 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.713814  1317 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.713907  1321 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.687288  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.712157   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:26.703835  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.720865  1077 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.720981  1002 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:26.714234  1317 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.725412   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.721645  1321 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.728408   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.728969   929 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:26.726883  1044 raft_consensus.cc:359] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.729622  1044 raft_consensus.cc:385] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.730024  1044 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.730876  1044 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.722465  1046 ts_tablet_manager.cc:1434] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:12:26.721254  1077 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.733326  1077 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.734376  1046 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:26.723014   750 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.724885  1045 tablet_bootstrap.cc:654] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.737980  1044 ts_tablet_manager.cc:1434] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:12:26.735780  1307 raft_consensus.cc:697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:26.753851  1077 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.749459  1307 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.758016  1321 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.762179   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.762732   929 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.742928   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:26.765738   849 raft_consensus.cc:2393] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:26.767686  1317 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.768855  1003 raft_consensus.cc:2393] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:26.771301  1044 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.772913  1321 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.774049   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:26.775894  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.776485  1003 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.777494   929 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.779179   818 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:26.780601  1240 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.781368  1240 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:26.782466  1240 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.783916  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:26.784808  1000 raft_consensus.cc:2393] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:26.774808  1077 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.789680  1046 tablet_bootstrap.cc:654] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.783314  1003 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.775203  1321 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.791525  1319 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.794847  1097 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.794359   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:26.795235   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:26.795457   929 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.795323  1097 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.796128   849 raft_consensus.cc:2393] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:26.797590  1097 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.798569   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.799067  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.799425   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:26.799716   895 leader_election.cc:343] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.799292  1317 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:26.800979   967 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:26.801081   896 leader_election.cc:343] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.801851   896 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.802667  1097 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.809001  1044 tablet_bootstrap.cc:654] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.803341  1321 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.813807   929 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:26.815040   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:26.816238  1240 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.816843  1240 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:26.817734  1240 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.821264  1045 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:26.821643  1045 ts_tablet_manager.cc:1403] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.178s	user 0.008s	sys 0.005s
I20260709 13:12:26.833858  1365 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.834723  1365 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.838723   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:26.841291  1365 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:26.851904   753 catalog_manager.cc:5697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.823918  1307 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:12:26.851418   750 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.859314  1317 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.863294  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:26.866739   999 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.870687  1045 raft_consensus.cc:359] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.871447  1045 raft_consensus.cc:385] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.871748  1045 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.872349  1045 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.889976  1045 ts_tablet_manager.cc:1434] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.000s
I20260709 13:12:26.890995  1045 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:26.890552  1097 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.896816   929 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.904791  1046 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:26.905289  1046 ts_tablet_manager.cc:1403] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.171s	user 0.024s	sys 0.018s
I20260709 13:12:26.908432  1321 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.910567   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:26.911953  1097 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.913964  1046 raft_consensus.cc:359] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.914746  1046 raft_consensus.cc:385] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.915057  1046 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.915800  1046 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.920614  1044 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:26.921084  1044 ts_tablet_manager.cc:1403] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.177s	user 0.010s	sys 0.004s
I20260709 13:12:26.923741  1044 raft_consensus.cc:359] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.924393  1044 raft_consensus.cc:385] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.924659  1044 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.925292  1044 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:26.927336  1044 ts_tablet_manager.cc:1434] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:26.930127  1044 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:26.932752   753 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.937294  1045 tablet_bootstrap.cc:654] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.949225  1046 ts_tablet_manager.cc:1434] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:12:26.952610  1044 tablet_bootstrap.cc:654] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.970157  1046 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.007035  1046 tablet_bootstrap.cc:654] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.020889  1045 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:27.021510  1045 ts_tablet_manager.cc:1403] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.131s	user 0.009s	sys 0.027s
I20260709 13:12:27.023193  1044 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.023677  1044 ts_tablet_manager.cc:1403] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.001s
I20260709 13:12:27.024801  1045 raft_consensus.cc:359] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.027774  1045 raft_consensus.cc:385] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.028069  1045 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.026155  1044 raft_consensus.cc:359] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.028525  1044 raft_consensus.cc:385] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.028964  1044 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.028731  1045 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.031103  1045 ts_tablet_manager.cc:1434] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:12:27.031481  1044 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.033744  1046 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.038193  1046 ts_tablet_manager.cc:1403] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.000s
I20260709 13:12:27.041356  1046 raft_consensus.cc:359] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.042563  1046 raft_consensus.cc:385] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.042924  1046 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.043705  1046 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.047173  1046 ts_tablet_manager.cc:1434] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:12:27.048255  1046 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.032150  1045 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:27.053545  1044 ts_tablet_manager.cc:1434] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:12:27.054556  1044 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.072876  1044 tablet_bootstrap.cc:654] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.072955   929 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:12:27.075404  1317 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.077692  1046 tablet_bootstrap.cc:654] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.081521  1045 tablet_bootstrap.cc:654] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.091796  1253 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.092407  1253 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.097483  1253 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.095515   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.099002  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:27.100271   896 leader_election.cc:343] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.102200   895 leader_election.cc:343] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.102577   895 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.104204  1307 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.107820  1044 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.108150  1046 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.108286  1044 ts_tablet_manager.cc:1403] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.006s
I20260709 13:12:27.108560  1046 ts_tablet_manager.cc:1403] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.061s	user 0.007s	sys 0.005s
I20260709 13:12:27.112015  1046 raft_consensus.cc:359] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.112653  1046 raft_consensus.cc:385] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.112706   852 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:12:27.112941  1046 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.113619  1046 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.114420  1241 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.115681  1046 ts_tablet_manager.cc:1434] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:12:27.118314  1044 raft_consensus.cc:359] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.118878  1046 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.119012  1044 raft_consensus.cc:385] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.119356  1044 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.120014  1044 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.125026  1046 tablet_bootstrap.cc:654] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.122943  1044 ts_tablet_manager.cc:1434] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:12:27.134407  1044 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.136196  1046 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.136673  1046 ts_tablet_manager.cc:1403] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 13:12:27.139237  1046 raft_consensus.cc:359] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.139966  1046 raft_consensus.cc:385] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.140224  1046 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.140918  1046 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.141741  1044 tablet_bootstrap.cc:654] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.142904  1046 ts_tablet_manager.cc:1434] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:27.145933  1046 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.154769  1046 tablet_bootstrap.cc:654] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.161720  1044 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.162070  1044 ts_tablet_manager.cc:1403] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.013s
I20260709 13:12:27.164804  1044 raft_consensus.cc:359] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.165482  1044 raft_consensus.cc:385] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.165741  1044 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.168111  1044 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.169984  1046 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.170461  1046 ts_tablet_manager.cc:1403] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.001s
I20260709 13:12:27.172151  1240 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.173105  1046 raft_consensus.cc:359] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.173763  1046 raft_consensus.cc:385] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.174048  1046 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.174683  1046 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.176780  1044 ts_tablet_manager.cc:1434] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:27.177264  1045 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:27.177738  1044 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.177771  1045 ts_tablet_manager.cc:1403] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.003s
I20260709 13:12:27.184295  1046 ts_tablet_manager.cc:1434] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:12:27.185253  1046 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.185518  1319 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.189385  1044 tablet_bootstrap.cc:654] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.189514  1045 raft_consensus.cc:359] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.191062  1240 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.195042  1046 tablet_bootstrap.cc:654] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.190227  1045 raft_consensus.cc:385] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.200182  1045 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.201354  1045 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.210480  1253 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.212746  1045 ts_tablet_manager.cc:1434] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:12:27.214090  1253 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.216634  1045 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
W20260709 13:12:27.239944   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:27.258242  1045 tablet_bootstrap.cc:654] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.263809  1044 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.264400  1044 ts_tablet_manager.cc:1403] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.087s	user 0.035s	sys 0.007s
I20260709 13:12:27.274278  1302 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.275349  1253 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.276062  1253 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.280000  1253 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.286296  1044 raft_consensus.cc:359] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.287068  1044 raft_consensus.cc:385] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.287344  1044 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.288064  1044 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.292243  1044 ts_tablet_manager.cc:1434] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:12:27.293260  1044 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.293475  1046 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.293979  1046 ts_tablet_manager.cc:1403] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.109s	user 0.035s	sys 0.026s
I20260709 13:12:27.296041   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.296268  1337 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.296708  1253 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.297176  1253 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.297945  1046 raft_consensus.cc:359] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.298722  1046 raft_consensus.cc:385] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.299005  1046 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.296749  1337 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.299752  1046 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.308817  1046 ts_tablet_manager.cc:1434] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:12:27.309214   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.299424  1253 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.312830  1044 tablet_bootstrap.cc:654] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.314307  1253 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.314908  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:27.318810   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.314499  1046 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.311304   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.312929  1337 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.316352  1241 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:27.317270   895 leader_election.cc:343] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.319470   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.324376   849 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.314920  1253 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.326443   969 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.327271  1317 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.327658  1317 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.328011  1317 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.331393  1046 tablet_bootstrap.cc:654] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:27.319494   896 leader_election.cc:343] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.332847   895 leader_election.cc:343] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.333602   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:27.335088   895 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.335474   999 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:27.335898  1097 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.336826   896 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:27.337638  1097 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.338023  1097 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.338331  1097 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.339732  1302 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.344854  1302 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.345309  1302 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.348908   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:27.349745  1302 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.350217  1317 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.324959  1241 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.353649  1241 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.355338   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.356282   849 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.356381   999 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:27.327262  1253 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.351280  1046 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.357726  1046 ts_tablet_manager.cc:1403] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.005s
I20260709 13:12:27.357982   969 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.358364   821 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.359011  1241 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.359767   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.359511  1302 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.360397  1302 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.360790  1302 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.360893  1046 raft_consensus.cc:359] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.361608  1046 raft_consensus.cc:385] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.362130  1046 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.324718  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:27.360412   927 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:27.359946  1241 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.365552  1241 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.366045   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.368765   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:27.368849   895 leader_election.cc:343] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.369349   927 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.363106  1046 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.374542  1046 ts_tablet_manager.cc:1434] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:12:27.381075  1302 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.380471  1241 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.384104  1302 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.385059  1337 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.385458  1045 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:27.385684  1337 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.386132  1045 ts_tablet_manager.cc:1403] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.172s	user 0.010s	sys 0.005s
I20260709 13:12:27.386320   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:27.387041   999 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:27.387780  1337 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.389334   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.395188   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.395419   852 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.395861   928 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.397055   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.390974  1319 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.399247   849 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.393726   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.399945   927 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
W20260709 13:12:27.394001   896 leader_election.cc:343] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.403352   896 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.393826  1097 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.397907   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.405002  1337 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.405418  1337 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.405481   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.406095  1337 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.406558   852 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:27.393463  1046 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.390272  1317 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.413113   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.413551  1241 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.415081   927 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.425380  1045 raft_consensus.cc:359] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.426419  1045 raft_consensus.cc:385] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.426723  1045 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.428612  1097 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.419205  1307 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.435729   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.438625  1046 tablet_bootstrap.cc:654] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.441354   849 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:12:27.453970   895 leader_election.cc:343] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.439481  1337 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.457938  1337 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.447255   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.460669   927 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.464282  1045 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.465444   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:27.466269  1044 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.498358  1045 ts_tablet_manager.cc:1434] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.112s	user 0.012s	sys 0.002s
W20260709 13:12:27.531472   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 66099.4us
I20260709 13:12:27.478338   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.535578   849 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.535815  1402 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.536674  1402 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:12:27.535429   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64029.9us
I20260709 13:12:27.540479  1044 ts_tablet_manager.cc:1403] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.247s	user 0.028s	sys 0.058s
I20260709 13:12:27.540779   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.478652   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.545821  1337 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.542622  1402 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.553566   927 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.563536  1044 raft_consensus.cc:359] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.564293  1045 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:27.577643   849 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.577957  1046 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.579012  1046 ts_tablet_manager.cc:1403] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.204s	user 0.011s	sys 0.012s
I20260709 13:12:27.572824  1044 raft_consensus.cc:385] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.579396   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.579587  1044 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.580438  1044 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.580786  1405 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.581521  1405 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:27.581650  1046 raft_consensus.cc:359] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.582327  1046 raft_consensus.cc:385] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.582653  1046 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.582743  1044 ts_tablet_manager.cc:1434] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.003s
I20260709 13:12:27.582369  1405 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.583881  1044 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.584154   927 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.583288  1046 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.592300  1045 tablet_bootstrap.cc:654] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.596850   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.597682   929 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:27.605221  1046 ts_tablet_manager.cc:1434] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:12:27.594451  1337 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:27.613371  1046 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.619863   818 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:27.621946  1337 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.630271   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:27.628826  1322 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.633759  1044 tablet_bootstrap.cc:654] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.635648   999 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.650859  1253 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.651402  1253 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.653520  1253 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:27.657184   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.633194  1408 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.658355  1408 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:27.659015   895 leader_election.cc:343] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.663250   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:27.664963   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:27.665609   849 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.667896   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.668977  1408 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.669461  1402 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.669827  1402 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.670195  1402 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:27.680234   896 leader_election.cc:343] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.681504   896 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.684271   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.689965   928 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.677790   999 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:27.697225  1407 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.701992   896 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.703176  1364 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.704622  1364 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:27.711190  1046 tablet_bootstrap.cc:654] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.707968  1364 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.713558   751 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.715215  1044 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.715915  1044 ts_tablet_manager.cc:1403] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.008s
I20260709 13:12:27.715216   751 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.717368   751 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.722680  1045 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:27.723220  1045 ts_tablet_manager.cc:1403] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.177s	user 0.005s	sys 0.013s
I20260709 13:12:27.725732  1402 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.726243  1045 raft_consensus.cc:359] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.726486  1044 raft_consensus.cc:359] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.727237  1044 raft_consensus.cc:385] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.726886  1045 raft_consensus.cc:385] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.727530  1044 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.727912  1045 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.728843  1044 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.729127  1045 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.731377  1044 ts_tablet_manager.cc:1434] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:12:27.731428   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.732101   849 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.732460  1044 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.741160  1044 tablet_bootstrap.cc:654] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.745246  1045 ts_tablet_manager.cc:1434] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:12:27.745805   849 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.746966   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:27.747867  1409 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.748317  1409 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:27.749082  1409 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.758860  1044 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.759284  1044 ts_tablet_manager.cc:1403] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.006s
I20260709 13:12:27.762027  1045 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:27.762080  1044 raft_consensus.cc:359] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.763031  1044 raft_consensus.cc:385] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.763324  1044 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.764040  1044 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.766172  1044 ts_tablet_manager.cc:1434] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:27.766423   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.767047  1044 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.767154   928 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.767210  1402 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.773903  1044 tablet_bootstrap.cc:654] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.786649   928 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.799876   752 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.804633  1046 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.805195  1046 ts_tablet_manager.cc:1403] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.197s	user 0.009s	sys 0.013s
I20260709 13:12:27.809201  1046 raft_consensus.cc:359] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.810005  1046 raft_consensus.cc:385] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.810349  1046 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.811152  1046 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.814463  1046 ts_tablet_manager.cc:1434] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:12:27.815455  1046 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.823452  1046 tablet_bootstrap.cc:654] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.845243  1046 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.845772  1046 ts_tablet_manager.cc:1403] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.010s
I20260709 13:12:27.848639  1046 raft_consensus.cc:359] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.849841  1046 raft_consensus.cc:385] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.850227  1046 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.850988  1046 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.853331  1046 ts_tablet_manager.cc:1434] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:27.854468  1046 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.855216  1409 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.855724  1409 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.857890  1409 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.860546  1046 tablet_bootstrap.cc:654] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.860366   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:27.861828   969 leader_election.cc:343] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.862481   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.863523   926 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.865609   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.867543  1402 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.868073  1402 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.868517  1402 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.894827  1045 tablet_bootstrap.cc:654] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.879360  1402 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.909439  1044 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.909924  1044 ts_tablet_manager.cc:1403] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.143s	user 0.041s	sys 0.069s
I20260709 13:12:27.912700  1044 raft_consensus.cc:359] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.913388  1044 raft_consensus.cc:385] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.913671  1044 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.914261  1402 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.914652   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:27.913929   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:27.916345   928 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:27.917855   969 leader_election.cc:343] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.914306  1044 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.920403  1044 ts_tablet_manager.cc:1434] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:12:27.921417  1044 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.925009  1045 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:27.925995  1045 ts_tablet_manager.cc:1403] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.164s	user 0.011s	sys 0.001s
I20260709 13:12:27.928416  1044 tablet_bootstrap.cc:654] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.928449  1045 raft_consensus.cc:359] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.934968  1045 raft_consensus.cc:385] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.935257  1045 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.937433  1409 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.937955  1409 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.938634  1045 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.940518  1045 ts_tablet_manager.cc:1434] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:27.941120  1409 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.941747   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:27.943010   969 leader_election.cc:343] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.944235  1045 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:27.944528   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:27.945135   926 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:27.950660  1045 tablet_bootstrap.cc:654] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.950779   967 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.952673  1409 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.953052  1409 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.953364  1409 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.955858  1044 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:27.961570  1044 ts_tablet_manager.cc:1403] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:12:27.965353  1046 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:27.965121  1044 raft_consensus.cc:359] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.966475  1044 raft_consensus.cc:385] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.967120  1044 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.967500   928 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:27.967795  1044 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.968674   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:27.968787  1409 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.969827  1044 ts_tablet_manager.cc:1434] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:12:27.969827  1241 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.970489  1241 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:27.970927  1044 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:27.971379  1241 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.967200  1046 ts_tablet_manager.cc:1403] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.113s	user 0.034s	sys 0.010s
I20260709 13:12:27.977591  1046 raft_consensus.cc:359] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.978938  1044 tablet_bootstrap.cc:654] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.981513  1409 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:27.983484   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:27.985231   969 leader_election.cc:343] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.983112  1046 raft_consensus.cc:385] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.986445  1046 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.987056  1046 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:27.989239  1046 ts_tablet_manager.cc:1434] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:12:27.990180  1046 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:27.996179  1046 tablet_bootstrap.cc:654] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.997937  1337 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.999086  1402 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.006744  1045 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:28.007390  1045 ts_tablet_manager.cc:1403] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.063s	user 0.024s	sys 0.003s
I20260709 13:12:28.011214  1045 raft_consensus.cc:359] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.012080  1045 raft_consensus.cc:385] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.012478  1045 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.013211  1045 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.014354   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:28.014958   928 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.016050  1045 ts_tablet_manager.cc:1434] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:28.017305  1045 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:28.020293  1364 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.020794  1364 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.021714  1046 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:28.022153  1046 ts_tablet_manager.cc:1403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 13:12:28.022960  1364 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.024070   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.024775   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:28.025655   896 leader_election.cc:343] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:28.026513   895 leader_election.cc:343] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.026957   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:28.027693  1253 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.039108   752 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:28.050652  1364 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.052029  1364 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.053977  1364 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.055938   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.056643   999 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:28.060513   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.062044   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:28.062922  1408 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.063699   895 leader_election.cc:343] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.063412  1408 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.057327  1046 raft_consensus.cc:359] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.064842  1046 raft_consensus.cc:385] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.065117  1046 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.067796  1407 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.068724  1407 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.069065  1407 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.069689  1045 tablet_bootstrap.cc:654] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.070881  1046 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.043876  1241 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:12:28.074817  1408 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:28.076718  1046 ts_tablet_manager.cc:1434] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.001s
I20260709 13:12:28.082659  1044 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:28.083320   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:28.084024   927 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:28.085067   967 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:28.089399   928 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:28.089609  1241 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:28.089803   956 tablet.cc:2406] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:12:28.090103  1241 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.090608  1241 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.093060   967 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:28.087736  1046 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:28.085875  1319 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.098685  1319 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.087724  1044 ts_tablet_manager.cc:1403] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.117s	user 0.021s	sys 0.012s
I20260709 13:12:28.104684  1364 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.114219  1364 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.114667  1321 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.115763  1402 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.116689   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.117313   855 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:28.115154  1321 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.149132  1319 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.149868  1253 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.150394  1253 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.176584   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:28.148226   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:28.203645   750 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:28.214848   930 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:28.215903  1321 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.230224  1241 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.227761  1307 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.228456  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.215065  1253 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.243301   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:28.252199  1046 tablet_bootstrap.cc:654] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.282708   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.225000  1405 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.222447  1402 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:28.248529  1408 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:12:28.261395  1319 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.291183  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.296267  1241 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:28.296950  1364 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.299178   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:28.300736   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.301234  1407 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.308465  1045 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:28.327695  1045 ts_tablet_manager.cc:1403] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.311s	user 0.028s	sys 0.020s
I20260709 13:12:28.283627   999 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:28.321061  1044 raft_consensus.cc:359] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.323441   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:28.329823  1044 raft_consensus.cc:385] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.329908   928 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.330324  1044 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.316962  1319 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.263871  1409 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 13:12:28.317718  1307 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.331159  1044 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:28.401829   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75659us
I20260709 13:12:28.405819  1044 ts_tablet_manager.cc:1434] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.305s	user 0.008s	sys 0.027s
I20260709 13:12:28.331780   847 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.407083   928 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:28.315158  1405 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.340313  1319 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:28.400905  1434 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:12:28.456499   821 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:28.409051  1307 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:28.463728   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:28.469762  1045 raft_consensus.cc:359] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.483805  1045 raft_consensus.cc:385] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.484187  1045 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.484961  1045 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.471700  1443 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.443434   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.410856  1402 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.478567  1405 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:28.449028  1001 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 0.
I20260709 13:12:28.495541  1307 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.497001  1044 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:28.498618  1407 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.502362  1319 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.511935  1405 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.514750  1045 ts_tablet_manager.cc:1434] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.187s	user 0.002s	sys 0.005s
W20260709 13:12:28.533203   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 129586us
W20260709 13:12:28.534612   895 leader_election.cc:343] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:28.535388   895 leader_election.cc:343] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.571170  1045 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:28.612990   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.567848  1319 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.530699  1405 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.570950  1443 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.626163   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:28.626169   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:28.642063   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 78043.2us
I20260709 13:12:28.643965  1402 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:28.646982  1448 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.647276  1046 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:28.647735  1046 ts_tablet_manager.cc:1403] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.565s	user 0.019s	sys 0.021s
I20260709 13:12:28.647500  1448 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.635428  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.639102  1447 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:12:28.654564   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.661741  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:28.662426  1000 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.665598  1046 raft_consensus.cc:359] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.666469  1046 raft_consensus.cc:385] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.666887  1046 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.652917   927 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:28.667582  1046 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.662438  1405 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:28.678254  1046 ts_tablet_manager.cc:1434] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:12:28.674784   850 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:12:28.680440   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:28.681075   921 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.682320   967 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:12:28.689407   896 leader_election.cc:343] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.689950  1044 tablet_bootstrap.cc:654] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.689845   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:12:28.692708   896 leader_election.cc:343] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.694340   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:28.694033   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:28.691784  1364 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.692960  1447 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.699003  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:28.709267  1045 tablet_bootstrap.cc:654] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:28.711141   969 consensus_peers.cc:597] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:28.699832  1407 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.715230   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:28.716001   997 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.706898  1046 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:28.699230   847 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:12:28.718204   896 leader_election.cc:343] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.717329  1447 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:28.738245   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:28.816377   922 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:12:28.853448  1459 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.126s
W20260709 13:12:28.858301  1460 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:12:28.840170  1447 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.893096   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:28.893954   922 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:28.897676  1365 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.898324  1365 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:28.902460  1365 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:28.903333  1461 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:12:28.901458   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:28.912268  1463 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.912729  1463 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.913128  1463 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.922237  1046 tablet_bootstrap.cc:654] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.925760  1448 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:28.959975  1044 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:28.996650  1044 ts_tablet_manager.cc:1403] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.500s	user 0.017s	sys 0.024s
W20260709 13:12:28.938599   895 leader_election.cc:343] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.998991   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:12:29.003934   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 65384.7us
W20260709 13:12:29.007190   895 leader_election.cc:343] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.993206   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:28.978051   921 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:29.004776  1364 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.002748  1000 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:29.024470  1442 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:12:29.028352  1440 raft_consensus.cc:2764] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:12:29.031975   997 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:29.041165   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:29.050194  1463 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.068218   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:29.026451   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:29.080401   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:12:29.075726   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 61050.8us
I20260709 13:12:29.085654   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.083354  1448 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.093080  1046 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:29.094647  1448 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:29.094771  1046 ts_tablet_manager.cc:1403] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.400s	user 0.011s	sys 0.004s
I20260709 13:12:29.089496  1365 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.113463  1045 tablet_bootstrap.cc:492] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:29.113972  1045 ts_tablet_manager.cc:1403] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.555s	user 0.040s	sys 0.048s
I20260709 13:12:29.115262  1046 raft_consensus.cc:359] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.116230  1046 raft_consensus.cc:385] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.116564  1046 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.116107   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:29.128830   969 leader_election.cc:343] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.112304  1365 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:29.140443  1046 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.155375  1457 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.137189  1409 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:12:29.146337  1463 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:29.174022   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 52151.7us
I20260709 13:12:29.180256  1046 ts_tablet_manager.cc:1434] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.000s
I20260709 13:12:29.138455  1459 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:12:29.122318  1448 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:29.189095   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51820.1us
W20260709 13:12:29.196442   895 leader_election.cc:343] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:29.199375  1468 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:12:29.210666  1046 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Bootstrap starting.
I20260709 13:12:29.213032  1044 raft_consensus.cc:359] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.220113  1044 raft_consensus.cc:385] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:29.220618   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56330.5us
I20260709 13:12:29.220736  1044 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.200431   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:29.237738   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68226.3us
I20260709 13:12:29.196168  1365 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.239547   924 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.316713  1045 raft_consensus.cc:359] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.317517  1045 raft_consensus.cc:385] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.317814  1045 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.356683  1045 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:29.316761  1481 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:12:29.365895   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 132978us
I20260709 13:12:29.443104   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:29.575783  1478 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:12:29.585462  1476 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:29.630859  1045 ts_tablet_manager.cc:1434] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.517s	user 0.005s	sys 0.004s
W20260709 13:12:29.631214  1045 ts_tablet_manager.cc:1467] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Tablet startup took 516ms
I20260709 13:12:29.630411   847 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.633090  1476 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.641886  1457 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.651535  1046 tablet_bootstrap.cc:654] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.651443  1451 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:29.646126  1476 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:29.652334  1451 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:29.638536  1045 ts_tablet_manager.cc:1469] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Trace:
0709 13:12:20.441358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.441486 (+   128us) service_pool.cc:224] Handling call
0709 13:12:20.443169 (+  1683us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:20.443268 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:20.453583 (+ 10315us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:20.470098 (+ 16515us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:20.470703 (+   605us) inbound_call.cc:177] Queueing success response
0709 13:12:28.531897 (+8061194us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:12:29.114311 (+582414us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:12:29.241467 (+127156us) tablet_replica.cc:261] Starting instrumentation
0709 13:12:29.249952 (+  8485us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6421,"tablet-open.queue_time_us":8061254}
I20260709 13:12:29.663030  1045 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:29.665731  1488 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.667163  1488 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.673568  1044 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.674916   997 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:29.676568  1044 ts_tablet_manager.cc:1434] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.680s	user 0.015s	sys 0.009s
W20260709 13:12:29.676908  1044 ts_tablet_manager.cc:1467] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Tablet startup took 679ms
I20260709 13:12:29.678364  1488 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:29.684028   847 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:12:29.677162  1044 ts_tablet_manager.cc:1469] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Trace:
0709 13:12:20.549910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.550055 (+   145us) service_pool.cc:224] Handling call
0709 13:12:20.551837 (+  1782us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:20.551921 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:20.564264 (+ 12343us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:20.583317 (+ 19053us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:20.583763 (+   446us) inbound_call.cc:177] Queueing success response
0709 13:12:28.496555 (+7912792us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:12:28.996985 (+500430us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:12:29.141357 (+144372us) tablet_replica.cc:261] Starting instrumentation
0709 13:12:29.141754 (+   397us) tablet_replica.cc:280] Starting consensus
0709 13:12:29.673339 (+531585us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b4c0049c7c0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d00e0ba20 00007f0d00e1ca29 00007f0d00de2644 00007f0d00e09f2d 00007f0d00e09b93 00007f0d02b3a18b 00007f0d02b3acb5 00007f0d02b35f30 00007f0d02b34c40 00007f0d06531b2e 00007f0d07b53d8a
Metrics: {"mutex_wait_us":13509,"raft.queue_time_us":1282,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"spinlock_wait_cycles":5878528,"tablet-open.queue_time_us":7912898}
I20260709 13:12:29.686296  1451 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:29.695130   924 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:29.711961  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:29.727566  1473 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.503s	user 0.006s	sys 0.003s
I20260709 13:12:29.735409  1439 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:29.755276  1044 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:29.736063  1439 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.758811  1253 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:29.791141   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:29.794307  1044 tablet_bootstrap.cc:654] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.794188   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:12:29.803260   896 leader_election.cc:343] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.790016  1253 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.806763  1253 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:29.808391  1467 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.808970  1467 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:29.809828  1467 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.813027   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:29.813318  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:29.835626  1045 tablet_bootstrap.cc:654] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.839069  1046 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: No bootstrap required, opened a new log
I20260709 13:12:29.839502  1046 ts_tablet_manager.cc:1403] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent bootstrapping tablet: real 0.651s	user 0.040s	sys 0.036s
I20260709 13:12:29.847323   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:29.837627  1439 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:29.867893  1044 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:29.868418  1044 ts_tablet_manager.cc:1403] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.113s	user 0.009s	sys 0.011s
I20260709 13:12:29.871896  1044 raft_consensus.cc:359] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.872579  1044 raft_consensus.cc:385] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.872867  1044 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.873502  1044 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.879423  1405 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:29.892131  1044 ts_tablet_manager.cc:1434] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:12:29.896039   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:29.893814  1046 raft_consensus.cc:359] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.897800  1046 raft_consensus.cc:385] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:12:29.896325   818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.018s	sys 0.022s
I20260709 13:12:29.898110  1046 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.900647   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:29.902030  1046 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:29.904657  1046 ts_tablet_manager.cc:1434] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.000s
W20260709 13:12:29.905696   818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.018s	sys 0.022s
W20260709 13:12:29.906023   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 715912us
I20260709 13:12:29.894599  1447 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:12:29.915794   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:29.928737   895 leader_election.cc:343] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.929292   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.932278   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.935894  1365 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:29.932509   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:29.931826  1307 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.942896  1045 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:29.941430   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:29.951057  1044 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:29.922626  1441 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:12:29.955231  1045 ts_tablet_manager.cc:1403] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.292s	user 0.021s	sys 0.013s
I20260709 13:12:29.972172   896 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.973587   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:29.978863   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.980906  1443 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:29.992722  1479 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.993273  1479 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.993710  1479 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.002774   895 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:29.996665  1253 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:30.011047  1253 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.011668  1253 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.014292  1253 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.006704  1451 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:30.017136  1045 raft_consensus.cc:359] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.019487   753 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:30.023767   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:30.018111  1045 raft_consensus.cc:385] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.026763  1045 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.033493  1045 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.025007  1407 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.044222   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:30.074193   895 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:30.076480  1503 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:12:30.038542  1476 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:12:30.049963  1407 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.081277  1407 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.082414   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:30.083088   852 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:30.084765   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:30.085590  1482 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.085978  1482 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.086320  1482 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.082441   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.094882   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.096498  1045 ts_tablet_manager.cc:1434] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.141s	user 0.006s	sys 0.002s
I20260709 13:12:30.098747   896 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.101876  1045 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
W20260709 13:12:30.104682   896 leader_election.cc:343] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.087334   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 119) took 1069 ms. Trace:
I20260709 13:12:30.107812   750 rpcz_store.cc:276] 0709 13:12:29.017650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:29.017790 (+   140us) service_pool.cc:224] Handling call
0709 13:12:30.087217 (+1069427us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5294,"reported_tablets":9}
I20260709 13:12:30.129501  1044 tablet_bootstrap.cc:654] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.100855  1482 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.138722  1488 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.139653  1488 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.152779  1488 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.156581  1482 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.160552  1452 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.169739  1452 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.171940  1452 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:30.172389  1045 tablet_bootstrap.cc:654] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.162956   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.173616   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:30.166235   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:30.174355   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:30.173736   999 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
W20260709 13:12:30.177099   896 leader_election.cc:343] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:30.176374   895 leader_election.cc:343] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:30.188400   969 leader_election.cc:343] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.192296   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:30.193810   851 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.175097  1479 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.201335   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.198908   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:30.205856  1488 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.207290  1453 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:30.259749  1488 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.270462  1488 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.292932   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:30.307315  1479 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.312727  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:30.313390  1001 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.316622  1479 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.331509  1476 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.330580   851 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:30.350109  1045 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
W20260709 13:12:30.350488   895 leader_election.cc:343] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.351006  1045 ts_tablet_manager.cc:1403] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.254s	user 0.012s	sys 0.012s
I20260709 13:12:30.351159   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:30.355278  1439 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.357726  1439 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:30.358592  1439 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.369519  1001 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:30.372093   896 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.376425  1044 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:30.376895  1044 ts_tablet_manager.cc:1403] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.426s	user 0.009s	sys 0.006s
I20260709 13:12:30.379403  1337 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.381407  1447 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.382337  1491 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.382903  1491 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:30.383841  1491 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.382987  1488 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.391052   930 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:30.395234  1488 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.396878  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:30.400868   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:30.403980  1044 raft_consensus.cc:359] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.406940  1044 raft_consensus.cc:385] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.407234  1044 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.408746  1044 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.408703   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:30.406208  1402 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:30.412714   895 leader_election.cc:343] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.413121  1044 ts_tablet_manager.cc:1434] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:12:30.437645  1044 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:30.525933  1045 raft_consensus.cc:359] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.527005  1045 raft_consensus.cc:385] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.527379  1045 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.528604  1045 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.531379  1045 ts_tablet_manager.cc:1434] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.180s	user 0.007s	sys 0.000s
I20260709 13:12:30.548388   849 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.555542  1044 tablet_bootstrap.cc:654] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.562433  1402 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.568969  1045 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:30.580057  1044 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:30.580524  1044 ts_tablet_manager.cc:1403] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.161s	user 0.009s	sys 0.004s
I20260709 13:12:30.608554  1044 raft_consensus.cc:359] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.609071  1044 raft_consensus.cc:385] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.609237  1044 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.609663  1044 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.618839  1044 ts_tablet_manager.cc:1434] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.005s
I20260709 13:12:30.619872  1044 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:30.606235  1001 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:30.629179  1045 tablet_bootstrap.cc:654] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.633308   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.634531  1488 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.635063  1488 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:30.636224  1488 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.637252  1467 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.637847  1467 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.642824  1467 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:30.643631   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:30.645417   969 leader_election.cc:343] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.646956   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:30.664037   923 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 0.
I20260709 13:12:30.672492   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.675194  1467 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.675578  1467 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.675951  1467 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.683298   750 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.684240  1467 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.684988   750 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:12:30.687233   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:30.687219   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:30.687235  1467 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:30.689803   930 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.692808   752 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 118) took 1548 ms. Trace:
I20260709 13:12:30.693146   752 rpcz_store.cc:276] 0709 13:12:29.155309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:29.155467 (+   158us) service_pool.cc:224] Handling call
0709 13:12:30.692749 (+1537282us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":11}
W20260709 13:12:30.689459   969 leader_election.cc:343] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.701364  1044 tablet_bootstrap.cc:654] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.711838   930 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:30.713169   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.714792  1467 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.715521  1467 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:30.716559  1467 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.725267   753 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:12:30.727422   753 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.747030  1045 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:30.747555  1045 ts_tablet_manager.cc:1403] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.179s	user 0.017s	sys 0.003s
I20260709 13:12:30.747805  1488 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.748292  1488 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.750448  1488 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.750229  1045 raft_consensus.cc:359] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.750988  1045 raft_consensus.cc:385] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.751257  1045 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.751847  1045 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.753470   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:30.756666   895 leader_election.cc:343] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.757556  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.758183  1001 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:30.759438   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.760535  1488 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.760950  1488 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.761447  1488 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.762255  1045 ts_tablet_manager.cc:1434] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:30.761056  1044 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:30.764070  1044 ts_tablet_manager.cc:1403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.144s	user 0.016s	sys 0.001s
I20260709 13:12:30.763228  1045 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:30.772475  1488 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.775854  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:30.776443  1001 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.777640   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:12:30.779049   895 leader_election.cc:343] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.780272  1044 raft_consensus.cc:359] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.781360  1044 raft_consensus.cc:385] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.781620  1044 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.782333  1044 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.785066   752 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.792392   752 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.794577  1044 ts_tablet_manager.cc:1434] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.002s
I20260709 13:12:30.794026   752 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.795713  1044 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Bootstrap starting.
I20260709 13:12:30.778811  1488 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:30.809101   896 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:30.825589  1044 tablet_bootstrap.cc:654] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.833959  1001 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:30.835429   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:30.836757  1451 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.837237  1451 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:30.838277  1451 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.845187  1045 tablet_bootstrap.cc:654] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.866851  1477 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.867594  1477 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.870054  1477 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:30.871470  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.877796   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:30.880304   818 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:30.885501   895 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:30.886579  1477 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.890044  1044 tablet_bootstrap.cc:492] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: No bootstrap required, opened a new log
I20260709 13:12:30.908111  1044 ts_tablet_manager.cc:1403] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Time spent bootstrapping tablet: real 0.113s	user 0.014s	sys 0.020s
I20260709 13:12:30.910880  1044 raft_consensus.cc:359] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.918567  1044 raft_consensus.cc:385] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.917913   752 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.918987  1044 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.919869   750 catalog_manager.cc:5697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.922679  1044 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.926374  1044 ts_tablet_manager.cc:1434] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 13:12:30.928535  1045 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:30.929023  1045 ts_tablet_manager.cc:1403] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.166s	user 0.010s	sys 0.004s
I20260709 13:12:30.932406  1045 raft_consensus.cc:359] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.933118  1045 raft_consensus.cc:385] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.933429  1045 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.934084  1045 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.936272  1045 ts_tablet_manager.cc:1434] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:30.942852  1045 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:30.949151  1045 tablet_bootstrap.cc:654] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.959270  1479 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.964819  1045 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:30.965351  1045 ts_tablet_manager.cc:1403] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.003s
I20260709 13:12:30.968132  1045 raft_consensus.cc:359] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.968888  1045 raft_consensus.cc:385] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.969213  1045 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.969955  1045 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.972679  1045 ts_tablet_manager.cc:1434] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:30.973702  1045 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:30.980953  1045 tablet_bootstrap.cc:654] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.984184  1477 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.984923  1477 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:30.988330   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:30.990475   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:30.991570   998 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 1: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:30.994232   821 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:30.995862  1484 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.005121  1045 tablet_bootstrap.cc:492] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:31.005649  1045 ts_tablet_manager.cc:1403] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.009s
I20260709 13:12:31.008397  1045 raft_consensus.cc:359] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.009079  1045 raft_consensus.cc:385] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.009349  1045 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.009961  1045 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.012478  1045 ts_tablet_manager.cc:1434] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:31.013463  1045 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:31.017907  1484 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.988440  1477 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:31.021660  1045 tablet_bootstrap.cc:654] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.120545  1488 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:12:31.148201   851 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.164757  1253 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.168452  1439 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.168987  1439 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.170953  1439 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:31.175760   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:31.177356   895 leader_election.cc:343] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:31.182159   801 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:31.186684  1045 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:31.187260  1045 ts_tablet_manager.cc:1403] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.174s	user 0.030s	sys 0.016s
W20260709 13:12:31.195375   895 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:31.191104  1045 raft_consensus.cc:359] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.197577  1045 raft_consensus.cc:385] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.197932  1045 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.200273  1045 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.204061  1045 ts_tablet_manager.cc:1434] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:12:31.204955  1045 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Bootstrap starting.
I20260709 13:12:31.208770  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:31.209465  1002 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 0.
I20260709 13:12:31.210901  1045 tablet_bootstrap.cc:654] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.215304   852 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.222491   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:31.224397  1467 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.225371  1483 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.225895  1045 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: No bootstrap required, opened a new log
I20260709 13:12:31.226075  1488 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.226414  1045 ts_tablet_manager.cc:1403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 13:12:31.226459  1488 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.226936  1488 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.229349  1045 raft_consensus.cc:359] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.230011  1045 raft_consensus.cc:385] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.230293  1045 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.230908  1045 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.233213  1045 ts_tablet_manager.cc:1434] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:31.236279  1488 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.240530  1488 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:31.248252  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:31.248867  1002 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.253707   997 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.267380   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:31.268343   852 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.273411   851 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.284561  1002 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:31.286002   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:31.286760  1253 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.287456  1253 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:31.288343  1253 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.292992  1490 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.306069   752 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.356731   852 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:31.417176  1253 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.525172  1477 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.525703  1477 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:31.528992  1477 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:31.529421   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:31.530387   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:31.531826   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:31.533205  1477 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.621920   855 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.778843  1253 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.790752  1451 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.870651 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:31.893484  1555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:31.893651  1556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:31.897313  1558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:31.897521 32224 server_base.cc:1061] running on GCE node
I20260709 13:12:31.899289 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:31.899569 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:31.899842 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602751899822 us; error 0 us; skew 500 ppm
I20260709 13:12:31.900525 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:31.906005 32224 webserver.cc:533] Webserver started at http://127.31.120.4:33055/ using document root <none> and password file <none>
I20260709 13:12:31.906737 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:31.907001 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:31.907424 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:31.909138 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-3-root/instance:
uuid: "597687e464174d1ea6043161e037d240"
format_stamp: "Formatted at 2026-07-09 13:12:31 on dist-test-slave-pm98"
I20260709 13:12:31.915915 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:12:31.920444  1563 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:31.921394 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:12:31.921708 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
uuid: "597687e464174d1ea6043161e037d240"
format_stamp: "Formatted at 2026-07-09 13:12:31 on dist-test-slave-pm98"
I20260709 13:12:31.921991 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:31.941685 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:31.943595 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:31.950601 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:31.956499 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:31.956765 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:31.957062 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:31.957285 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.146934 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.4:33689
I20260709 13:12:32.147534  1625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.4:33689 every 8 connection(s)
I20260709 13:12:32.153190 32224 auto_leader_rebalancer-test.cc:384] add a tserver: 597687e464174d1ea6043161e037d240
I20260709 13:12:32.173468  1626 heartbeater.cc:344] Connected to a master server at 127.31.120.62:45937
I20260709 13:12:32.173913  1626 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:32.174605  1626 heartbeater.cc:507] Master 127.31.120.62:45937 requested a full tablet report, sending...
I20260709 13:12:32.176666   752 ts_manager.cc:194] Registered new tserver with Master: 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:32.178647   752 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41676
I20260709 13:12:32.317271   923 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:32.326745  1001 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.328449  1439 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:32.331029   852 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.335162  1439 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:32.338807  1439 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.340832  1001 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.362977   752 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.395263   855 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.435420   997 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:12:32.440243   894 consensus_peers.cc:597] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82ed8549f06d4ef3a4620f596e0139b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:32.469976   855 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.471833  1467 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:32.472250   923 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.475056  1467 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:32.488917   968 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0cca622722b48a7ba611a3b77376f80. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:32.492543  1440 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.494366   855 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.519325   753 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.533375   930 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.540123   923 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:12:32.581576   894 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4997f2c7b33048a5b010a4e0c7866fa2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:32.582760   997 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.582921   855 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.584256  1439 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:32.585242  1635 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:32.592300  1635 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.601264   855 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.609650   998 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.610476  1001 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:32.616444   750 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:32.640929   968 consensus_peers.cc:597] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b4613bf5fb0446a80ac401fe0560967. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:32.641347   923 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.642876   855 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:32.644246  1483 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:32.645948  1483 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:32.664070  1467 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.673555   855 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.671223   923 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:32.685827   750 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.911213  1652 ts_tablet_manager.cc:933] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:32.916746  1654 ts_tablet_manager.cc:933] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:33.056195  1654 tablet_copy_client.cc:323] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:33.086678  1652 tablet_copy_client.cc:323] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:33.086968  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet e0cca622722b48a7ba611a3b77376f80 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:33.087561  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet e0cca622722b48a7ba611a3b77376f80 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-e0cca622722b48a7ba611a3b77376f80
I20260709 13:12:33.131755  1014 tablet_copy_source_session.cc:215] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:33.143833  1654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0cca622722b48a7ba611a3b77376f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.178442   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 82ed8549f06d4ef3a4620f596e0139b7 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:33.179086   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 82ed8549f06d4ef3a4620f596e0139b7 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-82ed8549f06d4ef3a4620f596e0139b7
I20260709 13:12:33.181452  1626 heartbeater.cc:499] Master 127.31.120.62:45937 was elected leader, sending a full tablet report...
I20260709 13:12:33.186936   940 tablet_copy_source_session.cc:215] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:33.190315  1652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ed8549f06d4ef3a4620f596e0139b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.253628  1654 tablet_copy_client.cc:806] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:33.258417  1654 tablet_copy_client.cc:670] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:33.263383  1660 ts_tablet_manager.cc:933] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:33.264881  1654 tablet_copy_client.cc:538] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:33.270694  1652 tablet_copy_client.cc:806] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:33.271325  1652 tablet_copy_client.cc:670] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:33.274145  1659 ts_tablet_manager.cc:933] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:33.278400  1652 tablet_copy_client.cc:538] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:33.293877  1660 tablet_copy_client.cc:323] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:33.296411  1659 tablet_copy_client.cc:323] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:33.301235  1654 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:33.298713  1652 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:33.301759   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 4997f2c7b33048a5b010a4e0c7866fa2 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:33.305827   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 4997f2c7b33048a5b010a4e0c7866fa2 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-4997f2c7b33048a5b010a4e0c7866fa2
I20260709 13:12:33.304265  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 2b4613bf5fb0446a80ac401fe0560967 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:33.309872  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 2b4613bf5fb0446a80ac401fe0560967 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-2b4613bf5fb0446a80ac401fe0560967
I20260709 13:12:33.334369  1014 tablet_copy_source_session.cc:215] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:33.339208  1659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4613bf5fb0446a80ac401fe0560967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.349037   940 tablet_copy_source_session.cc:215] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:33.355780  1660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4997f2c7b33048a5b010a4e0c7866fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.369858  1654 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:33.377769  1654 tablet_bootstrap.cc:492] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:33.379702  1654 ts_tablet_manager.cc:1403] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.079s	user 0.029s	sys 0.003s
I20260709 13:12:33.386421  1654 raft_consensus.cc:359] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.387409  1654 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:33.388305  1660 tablet_copy_client.cc:806] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:33.389197  1660 tablet_copy_client.cc:670] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:33.392947  1660 tablet_copy_client.cc:538] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:33.388156  1654 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.407805  1654 ts_tablet_manager.cc:1434] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.028s	user 0.013s	sys 0.000s
I20260709 13:12:33.413692  1660 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:33.422351  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-e0cca622722b48a7ba611a3b77376f80 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:33.422901  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-e0cca622722b48a7ba611a3b77376f80 on tablet e0cca622722b48a7ba611a3b77376f80 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:33.427420  1659 tablet_copy_client.cc:806] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:33.444238  1659 tablet_copy_client.cc:670] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:33.455098  1659 tablet_copy_client.cc:538] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:33.464073  1652 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:33.465444  1652 tablet_bootstrap.cc:492] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:33.471588  1659 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:33.472149  1652 ts_tablet_manager.cc:1403] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.174s	user 0.017s	sys 0.010s
I20260709 13:12:33.478827  1652 raft_consensus.cc:359] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.480435  1652 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:33.481161  1652 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.483204  1652 ts_tablet_manager.cc:1434] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:12:33.493132   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-82ed8549f06d4ef3a4620f596e0139b7 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:33.493646   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-82ed8549f06d4ef3a4620f596e0139b7 on tablet 82ed8549f06d4ef3a4620f596e0139b7 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:33.506067  1660 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:33.507206  1660 tablet_bootstrap.cc:492] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:33.510587  1601 raft_consensus.cc:1217] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:33.511588  1660 ts_tablet_manager.cc:1403] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.032s
I20260709 13:12:33.514848  1660 raft_consensus.cc:359] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.516892  1660 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:33.517604  1660 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.521526  1660 ts_tablet_manager.cc:1434] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:12:33.524844   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-4997f2c7b33048a5b010a4e0c7866fa2 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:33.525297   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-4997f2c7b33048a5b010a4e0c7866fa2 on tablet 4997f2c7b33048a5b010a4e0c7866fa2 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:33.536828  1659 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:33.537962  1659 tablet_bootstrap.cc:492] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:33.538758  1659 ts_tablet_manager.cc:1403] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.067s	user 0.029s	sys 0.004s
I20260709 13:12:33.541947  1659 raft_consensus.cc:359] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.542929  1659 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:33.543725  1659 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:33.546717  1659 ts_tablet_manager.cc:1434] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:33.548581  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-2b4613bf5fb0446a80ac401fe0560967 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:33.548969  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-2b4613bf5fb0446a80ac401fe0560967 on tablet 2b4613bf5fb0446a80ac401fe0560967 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:33.637214  1601 raft_consensus.cc:1217] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:33.657191  1601 raft_consensus.cc:1217] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:33.817868  1601 raft_consensus.cc:1217] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:34.078743  1467 raft_consensus.cc:1064] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:34.082157  1467 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:34.109525  1676 raft_consensus.cc:1064] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:34.110836  1601 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.112486  1440 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.119457  1676 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:34.128929   927 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.138741   855 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.141885  1669 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.151496  1600 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.153131  1439 raft_consensus.cc:1064] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:34.155062  1669 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.156258  1675 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.162354  1439 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:34.175033  1601 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.176760  1675 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:34.187913   855 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.189539  1668 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.204319  1001 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.205767   997 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.215633  1644 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.220809   846 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.250605   928 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.257522  1439 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.257732  1676 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:34.284854   750 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.297751  1669 raft_consensus.cc:1064] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:34.300166  1669 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:34.304363   851 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.326388  1600 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.326938   851 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.327973  1467 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.328838  1440 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:34.366591  1599 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.430887  1674 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:12:34.462966  1690 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:12:34.628901   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 237202us
I20260709 13:12:34.669304  1599 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.669304   998 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:34.667750  1451 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.618176  1668 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.775404  1601 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:12:34.780303   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 399800us
I20260709 13:12:34.795287   930 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:34.790169   997 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.820293  1440 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:34.826524   852 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.835491   928 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:34.912529   855 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.918910   753 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.853735  1601 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.920750   752 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:34.823055  1700 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.891927  1483 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.965509  1600 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:34.997952   750 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:35.028797   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 91322.7us
I20260709 13:12:35.028478  1599 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.054442  1669 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:35.073879  1710 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:12:35.077850   852 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.091147   999 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.123445   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2b4613bf5fb0446a80ac401fe0560967 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:35.122735   923 raft_consensus.cc:2955] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.141608   751 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:35.145828   921 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.191643  1001 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:35.193089   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 2b4613bf5fb0446a80ac401fe0560967 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:35.201961   921 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:35.204720  1600 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.203301   852 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.216516  1713 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:35.216856   923 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:35.218708  1713 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:35.220300   852 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.222136  1702 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:35.226567  1600 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.228893  1674 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:35.237776  1483 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.240199  1711 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.240365  1599 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.253463  1718 tablet_replica.cc:333] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:35.254139   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e0cca622722b48a7ba611a3b77376f80 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:35.258562  1718 raft_consensus.cc:2243] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:35.261047  1718 raft_consensus.cc:2272] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:35.265009   855 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.267031   852 raft_consensus.cc:2955] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.277132  1718 ts_tablet_manager.cc:1916] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:35.280256  1601 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.280675  1001 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:35.280973   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4997f2c7b33048a5b010a4e0c7866fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:35.281816  1674 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:35.288415  1702 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:35.269089  1600 raft_consensus.cc:2955] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.291577   751 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:35.292498  1698 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.307549   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 4997f2c7b33048a5b010a4e0c7866fa2 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:35.319783   999 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.329960  1601 raft_consensus.cc:2955] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.334897  1718 ts_tablet_manager.cc:1929] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:35.335529  1718 log.cc:1199] T 2b4613bf5fb0446a80ac401fe0560967 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/2b4613bf5fb0446a80ac401fe0560967
I20260709 13:12:35.345072   751 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:35.352069   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 2b4613bf5fb0446a80ac401fe0560967 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:35.378407   919 tablet_service.cc:1558] Processing DeleteTablet for tablet e0cca622722b48a7ba611a3b77376f80 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:35.388087   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82ed8549f06d4ef3a4620f596e0139b7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:35.396450  1719 tablet_replica.cc:333] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:35.399528   752 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:35.412837  1720 tablet_replica.cc:333] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:35.414017  1720 raft_consensus.cc:2243] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:35.414788  1720 raft_consensus.cc:2272] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:35.416163  1719 raft_consensus.cc:2243] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:35.418365  1719 raft_consensus.cc:2272] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:35.420732   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 82ed8549f06d4ef3a4620f596e0139b7 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:35.423561  1718 tablet_replica.cc:333] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:35.424659  1718 raft_consensus.cc:2243] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:35.425350  1718 raft_consensus.cc:2272] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:35.428803  1720 ts_tablet_manager.cc:1916] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:35.431406  1718 ts_tablet_manager.cc:1916] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:35.442504  1719 ts_tablet_manager.cc:1916] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:35.469693  1718 ts_tablet_manager.cc:1929] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:35.470301  1718 log.cc:1199] T 82ed8549f06d4ef3a4620f596e0139b7 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/82ed8549f06d4ef3a4620f596e0139b7
I20260709 13:12:35.472034  1720 ts_tablet_manager.cc:1929] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:35.472683   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 82ed8549f06d4ef3a4620f596e0139b7 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:35.473505  1720 log.cc:1199] T e0cca622722b48a7ba611a3b77376f80 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/e0cca622722b48a7ba611a3b77376f80
I20260709 13:12:35.479786   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet e0cca622722b48a7ba611a3b77376f80 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:35.483927  1719 ts_tablet_manager.cc:1929] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:35.485474  1719 log.cc:1199] T 4997f2c7b33048a5b010a4e0c7866fa2 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/4997f2c7b33048a5b010a4e0c7866fa2
I20260709 13:12:35.487845   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 4997f2c7b33048a5b010a4e0c7866fa2 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
W20260709 13:12:36.498194   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 233766us
W20260709 13:12:36.763417   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 504967us
W20260709 13:12:36.799398   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44890us
W20260709 13:12:36.854532   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 90424.4us
I20260709 13:12:36.861946   855 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:36.917630   999 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:36.978935  1724 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:36.998251   926 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:37.040650   819 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c43ab43025004edcacdb3d8a30106319. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.040911  1725 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.065070  1732 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.068251   999 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.089443   852 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.091837   752 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:37.097302   819 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6e1447d2c0c4822aac226064c087c42. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.098598   924 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.101055   995 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.103446  1724 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.101018   929 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.112723  1732 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.189380  1732 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.204838   930 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.231369   995 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.251781   846 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.261422   753 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:37.282049   819 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91555b4ec53349c287c5221b9030c23a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.287003   924 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.288973  1727 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.313016   995 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.318881  1727 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.393555  1732 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.397836  1001 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.415488   930 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.417079   924 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.423492   751 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:37.444527   894 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 24f44707b1d3466e91403f063187a9b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.445051  1755 ts_tablet_manager.cc:933] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:37.449640   995 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.452945  1736 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.449934   846 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.461284  1736 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.475669  1755 tablet_copy_client.cc:323] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:12:37.482048  1668 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.489722   997 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.505411   753 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:37.514106   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet c43ab43025004edcacdb3d8a30106319 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:12:37.514784   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet c43ab43025004edcacdb3d8a30106319 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-c43ab43025004edcacdb3d8a30106319
I20260709 13:12:37.523059   865 tablet_copy_source_session.cc:215] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:37.527040  1755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c43ab43025004edcacdb3d8a30106319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.554760  1755 tablet_copy_client.cc:806] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:37.555624  1755 tablet_copy_client.cc:670] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:37.559239   852 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:37.561373  1755 tablet_copy_client.cc:538] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:37.592124  1755 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:37.653515  1762 ts_tablet_manager.cc:933] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:37.663041  1762 tablet_copy_client.cc:323] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:12:37.669262   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet c6e1447d2c0c4822aac226064c087c42 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:12:37.669998   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet c6e1447d2c0c4822aac226064c087c42 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-c6e1447d2c0c4822aac226064c087c42
W20260709 13:12:37.672420   951 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6240 after lost signal to thread 32227
W20260709 13:12:37.673473   951 debug-util.cc:398] Leaking SignalData structure 0x7b08001f15a0 after lost signal to thread 787
W20260709 13:12:37.675410   951 debug-util.cc:398] Leaking SignalData structure 0x7b0800170d20 after lost signal to thread 954
W20260709 13:12:37.676934   951 debug-util.cc:398] Leaking SignalData structure 0x7b080030e260 after lost signal to thread 1028
I20260709 13:12:37.678098   865 tablet_copy_source_session.cc:215] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:37.703944  1762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e1447d2c0c4822aac226064c087c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.701714  1755 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:37.723767  1755 tablet_bootstrap.cc:492] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:37.724808  1755 ts_tablet_manager.cc:1403] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.154s	user 0.026s	sys 0.027s
I20260709 13:12:37.728379  1755 raft_consensus.cc:359] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.729348  1755 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:37.730120  1755 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.748196  1762 tablet_copy_client.cc:806] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:37.750110  1762 tablet_copy_client.cc:670] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:37.759371  1762 tablet_copy_client.cc:538] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:37.772119  1762 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:37.792606  1755 ts_tablet_manager.cc:1434] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.004s
I20260709 13:12:37.799502   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-c43ab43025004edcacdb3d8a30106319 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:12:37.800031   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-c43ab43025004edcacdb3d8a30106319 on tablet c43ab43025004edcacdb3d8a30106319 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:37.831418  1762 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:37.832729  1755 ts_tablet_manager.cc:933] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:37.833316  1762 tablet_bootstrap.cc:492] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:37.834117  1762 ts_tablet_manager.cc:1403] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.016s
I20260709 13:12:37.845721  1762 raft_consensus.cc:359] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.850773  1762 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:37.851567  1762 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:37.860935  1600 raft_consensus.cc:1217] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:37.868021  1755 tablet_copy_client.cc:323] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:12:37.871044   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet 91555b4ec53349c287c5221b9030c23a from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:12:37.871589   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet 91555b4ec53349c287c5221b9030c23a from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-91555b4ec53349c287c5221b9030c23a
I20260709 13:12:37.897256   865 tablet_copy_source_session.cc:215] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:37.900491  1755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91555b4ec53349c287c5221b9030c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.904966  1762 ts_tablet_manager.cc:1434] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.069s	user 0.004s	sys 0.002s
I20260709 13:12:37.919308   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-c6e1447d2c0c4822aac226064c087c42 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:12:37.919868   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-c6e1447d2c0c4822aac226064c087c42 on tablet c6e1447d2c0c4822aac226064c087c42 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:37.930716  1755 tablet_copy_client.cc:806] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:37.931483  1755 tablet_copy_client.cc:670] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:37.935456  1755 tablet_copy_client.cc:538] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:37.945775  1755 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:37.994297  1755 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:37.997164  1755 tablet_bootstrap.cc:492] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:37.997989  1755 ts_tablet_manager.cc:1403] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.014s
I20260709 13:12:38.005908  1755 raft_consensus.cc:359] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:38.006923  1755 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:38.007688  1755 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:38.010671  1755 ts_tablet_manager.cc:1434] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:12:38.012931   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-91555b4ec53349c287c5221b9030c23a received from {username='slave'} at 127.0.0.1:43438
I20260709 13:12:38.013381   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-91555b4ec53349c287c5221b9030c23a on tablet 91555b4ec53349c287c5221b9030c23a with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:38.024816  1755 ts_tablet_manager.cc:933] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:38.033097  1755 tablet_copy_client.cc:323] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:38.035501   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 24f44707b1d3466e91403f063187a9b2 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:38.036134   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 24f44707b1d3466e91403f063187a9b2 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-24f44707b1d3466e91403f063187a9b2
I20260709 13:12:38.046900   940 tablet_copy_source_session.cc:215] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:38.050609  1755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f44707b1d3466e91403f063187a9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:38.070459  1755 tablet_copy_client.cc:806] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:38.071197  1755 tablet_copy_client.cc:670] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:38.075977  1755 tablet_copy_client.cc:538] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:38.089708  1755 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:38.131253  1755 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:38.146250  1755 tablet_bootstrap.cc:492] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:38.147379  1755 ts_tablet_manager.cc:1403] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.058s	user 0.033s	sys 0.000s
I20260709 13:12:38.156136  1755 raft_consensus.cc:359] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:38.157028  1755 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:38.157686  1755 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:38.167969  1755 ts_tablet_manager.cc:1434] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:12:38.171312   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-24f44707b1d3466e91403f063187a9b2 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:38.171878   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-24f44707b1d3466e91403f063187a9b2 on tablet 24f44707b1d3466e91403f063187a9b2 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:38.240886  1600 raft_consensus.cc:1217] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.260502  1727 raft_consensus.cc:1064] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:38.267527  1727 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:38.367929  1600 raft_consensus.cc:1217] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.433382  1600 raft_consensus.cc:1217] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.447883   930 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:38.467068  1600 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:38.472754   997 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:38.489061  1777 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:38.497303  1732 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:38.547763  1724 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:38.623344  1724 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.637822   924 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.644492  1732 raft_consensus.cc:1064] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:38.646950  1732 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:38.666664   751 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.714604  1600 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:38.716367  1727 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:38.725922   995 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.767007   927 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:38.768510  1775 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:38.783798   996 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:12:38.790344  1782 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:12:38.801151  1600 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.815357  1776 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.833053   924 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.884352  1600 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.920367  1724 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:38.973569   995 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:38.978379  1777 raft_consensus.cc:1064] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:38.981160  1777 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:39.015378   752 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.047498  1774 raft_consensus.cc:1064] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:39.078253  1774 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:12:39.080296   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65490.5us
I20260709 13:12:39.141423  1600 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.142871  1775 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.149765   930 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.152124  1775 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.170904   852 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:39.221345  1600 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:12:39.226375   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50238.7us
I20260709 13:12:39.227758  1775 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.231523   927 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.239562  1775 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.240260  1600 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.259028  1777 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.261767   995 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.274350  1776 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.276515  1600 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.278039  1790 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.273993   753 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.297047  1600 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.301945   927 raft_consensus.cc:2955] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.309974  1776 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:39.313443   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c43ab43025004edcacdb3d8a30106319 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:39.315668   850 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.321460   752 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:39.323527   994 tablet_service.cc:1558] Processing DeleteTablet for tablet c43ab43025004edcacdb3d8a30106319 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:39.332931  1790 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.344758  1796 tablet_replica.cc:333] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:39.351653  1796 raft_consensus.cc:2243] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:39.351792   924 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.357396   995 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.358584   996 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.390041  1774 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:39.406881  1796 raft_consensus.cc:2272] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:39.409683  1797 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:39.419386   852 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:39.428269  1790 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.432006  1600 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.433852  1732 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.435976  1001 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.416684  1796 ts_tablet_manager.cc:1916] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:39.437703  1776 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.441265  1724 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.444710   853 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.455214  1001 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.453670  1600 raft_consensus.cc:2955] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.477959   752 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.484113  1600 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.495944   919 tablet_service.cc:1558] Processing DeleteTablet for tablet c6e1447d2c0c4822aac226064c087c42 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:39.503446   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c6e1447d2c0c4822aac226064c087c42 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:39.511960   753 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.546870   997 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.548338  1796 ts_tablet_manager.cc:1929] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:39.548882  1796 log.cc:1199] T c43ab43025004edcacdb3d8a30106319 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/c43ab43025004edcacdb3d8a30106319
I20260709 13:12:39.551016   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet c43ab43025004edcacdb3d8a30106319 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:39.556059   924 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:39.564074  1600 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.557729  1800 tablet_replica.cc:333] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:39.564471   853 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.565740  1800 raft_consensus.cc:2243] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:39.566557  1800 raft_consensus.cc:2272] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:39.567183  1668 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.569299  1774 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.576136  1800 ts_tablet_manager.cc:1916] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:39.577961  1668 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.579589   852 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.585253  1600 raft_consensus.cc:2955] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.594411  1800 ts_tablet_manager.cc:1929] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:39.594978  1800 log.cc:1199] T c6e1447d2c0c4822aac226064c087c42 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/c6e1447d2c0c4822aac226064c087c42
I20260709 13:12:39.596895   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet c6e1447d2c0c4822aac226064c087c42 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:39.604188   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 24f44707b1d3466e91403f063187a9b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:39.610751   750 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.646214   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 24f44707b1d3466e91403f063187a9b2 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:39.647210  1796 tablet_replica.cc:333] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:39.648223  1796 raft_consensus.cc:2243] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:39.648900  1796 raft_consensus.cc:2272] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:39.655490  1796 ts_tablet_manager.cc:1916] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:39.677341  1796 ts_tablet_manager.cc:1929] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:39.677736  1796 log.cc:1199] T 24f44707b1d3466e91403f063187a9b2 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/24f44707b1d3466e91403f063187a9b2
I20260709 13:12:39.679674   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 24f44707b1d3466e91403f063187a9b2 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:39.874042   852 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:39.881623  1600 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.883400  1724 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.892856   997 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.896148  1776 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.900144  1600 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:39.912226  1732 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.917354   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91555b4ec53349c287c5221b9030c23a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:39.924566  1001 raft_consensus.cc:2955] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:40.012383   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 77898.9us
I20260709 13:12:40.021086   750 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:40.043675   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 91555b4ec53349c287c5221b9030c23a with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:40.044633  1800 tablet_replica.cc:333] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:40.045729  1800 raft_consensus.cc:2243] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:40.047302  1800 raft_consensus.cc:2272] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:40.051244  1800 ts_tablet_manager.cc:1916] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:40.068145  1800 ts_tablet_manager.cc:1929] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:40.068562  1800 log.cc:1199] T 91555b4ec53349c287c5221b9030c23a P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/91555b4ec53349c287c5221b9030c23a
I20260709 13:12:40.069933   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 91555b4ec53349c287c5221b9030c23a (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:41.151314   928 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.160785   996 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.164816  1668 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:41.167526   894 consensus_peers.cc:597] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8772f5977d4b4dc6b9f7f5dcf7019b90. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.171969   852 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.175683  1635 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:41.190083  1668 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.200654   996 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.216485  1001 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.210716   852 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.221266   753 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:41.258551   968 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 516d927af6bf46aa947c0919f1dfc830. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.277570   852 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.279317  1729 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.289630   924 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.293193  1801 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:41.321424  1801 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.325645   851 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.326360   924 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.362720   753 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.366602   924 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.384023   851 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:41.384079   894 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6cfaab49f56546a49978c6055f57640f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.385571  1001 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.385468  1668 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.386972  1806 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.407847  1635 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.415691  1001 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.419139   852 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.443674  1001 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.435389   752 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:41.483100   924 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.484777  1801 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.485500   852 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.490088  1803 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:12:41.505658   968 consensus_peers.cc:597] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9794c08fc6b2428b981acd6c3c238b5f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.507341  1803 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.510074   924 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.512912   852 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:41.539499   750 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:41.638485  1822 ts_tablet_manager.cc:933] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:41.650849  1822 tablet_copy_client.cc:323] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:41.654043   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:41.654605   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-8772f5977d4b4dc6b9f7f5dcf7019b90
I20260709 13:12:41.662235   940 tablet_copy_source_session.cc:215] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:41.665479  1822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:41.684214  1822 tablet_copy_client.cc:806] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:41.684784  1822 tablet_copy_client.cc:670] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:41.691004  1822 tablet_copy_client.cc:538] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:41.699098  1822 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:41.738015  1822 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:41.738996  1822 tablet_bootstrap.cc:492] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:41.739750  1822 ts_tablet_manager.cc:1403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.015s
I20260709 13:12:41.742836  1822 raft_consensus.cc:359] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.743748  1822 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:41.744431  1822 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.754907  1822 ts_tablet_manager.cc:1434] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:12:41.758351   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-8772f5977d4b4dc6b9f7f5dcf7019b90 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:41.758821   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-8772f5977d4b4dc6b9f7f5dcf7019b90 on tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:41.779929  1822 ts_tablet_manager.cc:933] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:41.797358  1822 tablet_copy_client.cc:323] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:41.799439  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 516d927af6bf46aa947c0919f1dfc830 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:41.800068  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 516d927af6bf46aa947c0919f1dfc830 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-516d927af6bf46aa947c0919f1dfc830
I20260709 13:12:41.812160  1014 tablet_copy_source_session.cc:215] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:41.815460  1822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516d927af6bf46aa947c0919f1dfc830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:41.831882  1822 tablet_copy_client.cc:806] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:41.832515  1822 tablet_copy_client.cc:670] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:41.836357  1822 tablet_copy_client.cc:538] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:41.848867  1822 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:41.890559  1822 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:41.891464  1822 tablet_bootstrap.cc:492] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:41.892140  1822 ts_tablet_manager.cc:1403] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.043s	user 0.026s	sys 0.001s
I20260709 13:12:41.898031  1822 raft_consensus.cc:359] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.898855  1822 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:41.903256  1822 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:41.906010  1822 ts_tablet_manager.cc:1434] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:12:41.909902  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-516d927af6bf46aa947c0919f1dfc830 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:41.910382  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-516d927af6bf46aa947c0919f1dfc830 on tablet 516d927af6bf46aa947c0919f1dfc830 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:41.916807  1822 ts_tablet_manager.cc:933] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:41.936398  1822 tablet_copy_client.cc:323] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:41.938141  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 9794c08fc6b2428b981acd6c3c238b5f from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:41.938750  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 9794c08fc6b2428b981acd6c3c238b5f from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-9794c08fc6b2428b981acd6c3c238b5f
I20260709 13:12:41.948498  1014 tablet_copy_source_session.cc:215] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:41.951683  1822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9794c08fc6b2428b981acd6c3c238b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:41.968029  1822 tablet_copy_client.cc:806] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:41.968566  1822 tablet_copy_client.cc:670] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:41.975397  1822 tablet_copy_client.cc:538] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:41.994824  1830 ts_tablet_manager.cc:933] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:42.013800  1822 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:42.047672  1822 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.048635  1822 tablet_bootstrap.cc:492] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:42.049342  1822 ts_tablet_manager.cc:1403] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.013s
I20260709 13:12:42.062608  1822 raft_consensus.cc:359] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:42.063483  1822 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:42.064141  1822 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:42.074337  1822 ts_tablet_manager.cc:1434] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:12:42.075459  1830 tablet_copy_client.cc:323] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:42.077488   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 6cfaab49f56546a49978c6055f57640f from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:42.078037   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 6cfaab49f56546a49978c6055f57640f from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-6cfaab49f56546a49978c6055f57640f
I20260709 13:12:42.081521  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-9794c08fc6b2428b981acd6c3c238b5f received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:42.081995  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-9794c08fc6b2428b981acd6c3c238b5f on tablet 9794c08fc6b2428b981acd6c3c238b5f with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:42.088603   940 tablet_copy_source_session.cc:215] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:42.094940  1830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cfaab49f56546a49978c6055f57640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:42.116531  1830 tablet_copy_client.cc:806] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:42.117093  1830 tablet_copy_client.cc:670] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:42.121344  1830 tablet_copy_client.cc:538] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:42.130119  1830 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:42.179008  1600 raft_consensus.cc:1217] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.190294  1830 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.191323  1830 tablet_bootstrap.cc:492] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:42.192111  1830 ts_tablet_manager.cc:1403] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.062s	user 0.032s	sys 0.014s
I20260709 13:12:42.195595  1830 raft_consensus.cc:359] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:42.196486  1830 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:42.197160  1830 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:42.206830  1830 ts_tablet_manager.cc:1434] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:42.213900   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-6cfaab49f56546a49978c6055f57640f received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:42.214334   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-6cfaab49f56546a49978c6055f57640f on tablet 6cfaab49f56546a49978c6055f57640f with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:42.287814  1600 raft_consensus.cc:1217] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.487473  1600 raft_consensus.cc:1217] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.508483  1600 raft_consensus.cc:1217] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.711074  1835 raft_consensus.cc:1064] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:42.714042  1835 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:42.731329   924 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.731329  1600 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.739358  1803 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:12:42.746070  1729 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:42.751588   851 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.754056  1801 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:42.764966  1635 raft_consensus.cc:1064] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:42.767491  1635 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:42.811235  1835 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.819986   853 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.843629   753 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.856560   924 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.867215  1600 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.869557  1807 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:42.875811  1601 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.885174   996 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.895130  1668 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:42.898948   853 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:42.901158  1807 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:42.919433  1807 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.921017   996 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.933702   850 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.941411  1601 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.973551   996 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:42.975554   750 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:42.987380   850 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:42.989132  1601 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:42.996630  1835 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.003825  1835 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.008519  1801 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.012657   850 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.015677  1600 raft_consensus.cc:2955] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.019528  1845 raft_consensus.cc:1064] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:43.022178  1845 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:43.026919   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 516d927af6bf46aa947c0919f1dfc830 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.035619  1803 raft_consensus.cc:1064] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:43.038213  1803 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:43.055791   924 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:43.069869  1600 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.072417  1807 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.083945  1635 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.093772  1601 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.097195  1600 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.100270   996 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.100430  1729 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:43.103552  1806 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.113215   853 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.116474  1729 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.124300   927 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.134370  1729 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.147737   750 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.158030   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.167846   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 516d927af6bf46aa947c0919f1dfc830 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:43.185675   996 raft_consensus.cc:2955] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.198275  1857 tablet_replica.cc:333] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:43.197842  1819 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:43.204425  1853 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:12:43.201085   927 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.206607  1857 raft_consensus.cc:2243] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.199041   750 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.216722   852 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.219931  1600 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.244789   753 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:43.257755   997 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.259357   852 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.260149  1807 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.276139   997 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:43.272429  1857 raft_consensus.cc:2272] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.286835  1601 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.286873   924 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.287695  1806 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.289592  1835 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:12:43.293798  1856 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.005s
I20260709 13:12:43.273372  1600 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.297087  1845 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.297546  1801 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.305078  1857 ts_tablet_manager.cc:1916] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.316533  1801 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.318230   927 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.322688   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:43.346246  1668 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.349924  1600 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.354192   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9794c08fc6b2428b981acd6c3c238b5f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.363163  1599 raft_consensus.cc:2955] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.370322   750 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.376820   852 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.379755  1862 tablet_replica.cc:333] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:43.380954  1862 raft_consensus.cc:2243] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.381659  1862 raft_consensus.cc:2272] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.385450  1862 ts_tablet_manager.cc:1916] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.391753   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 9794c08fc6b2428b981acd6c3c238b5f with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:43.405179   750 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:43.425391  1862 ts_tablet_manager.cc:1929] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.424906   924 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:43.425925  1862 log.cc:1199] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/8772f5977d4b4dc6b9f7f5dcf7019b90
I20260709 13:12:43.427785   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 8772f5977d4b4dc6b9f7f5dcf7019b90 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:43.427922  1862 tablet_replica.cc:333] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:43.433612  1857 ts_tablet_manager.cc:1929] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.434060  1857 log.cc:1199] T 516d927af6bf46aa947c0919f1dfc830 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/516d927af6bf46aa947c0919f1dfc830
I20260709 13:12:43.440404  1600 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.445052  1668 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.445739   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 516d927af6bf46aa947c0919f1dfc830 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:43.456333  1807 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.457558  1601 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.466773   852 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.481206  1668 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.498461   852 raft_consensus.cc:2955] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.504403   751 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:43.515482   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 6cfaab49f56546a49978c6055f57640f with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:43.517571   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6cfaab49f56546a49978c6055f57640f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.526683  1862 raft_consensus.cc:2243] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.527510  1862 raft_consensus.cc:2272] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.537057  1862 ts_tablet_manager.cc:1916] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.552839  1864 tablet_replica.cc:333] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:43.554071  1864 raft_consensus.cc:2243] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.554698  1864 pending_rounds.cc:70] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Trying to abort 1 pending ops.
I20260709 13:12:43.555065  1864 pending_rounds.cc:77] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305636917332848640 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "6cfaab49f56546a49978c6055f57640f" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } } }
I20260709 13:12:43.556540  1864 raft_consensus.cc:2917] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:12:43.557017  1864 raft_consensus.cc:2889] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:12:43.557760  1864 raft_consensus.cc:2272] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.560410  1862 ts_tablet_manager.cc:1929] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.562085  1864 ts_tablet_manager.cc:1916] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.582180  1864 ts_tablet_manager.cc:1929] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.582706  1864 log.cc:1199] T 6cfaab49f56546a49978c6055f57640f P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/6cfaab49f56546a49978c6055f57640f
I20260709 13:12:43.585011   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 6cfaab49f56546a49978c6055f57640f (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:43.560952  1862 log.cc:1199] T 9794c08fc6b2428b981acd6c3c238b5f P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/9794c08fc6b2428b981acd6c3c238b5f
I20260709 13:12:43.627799   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 9794c08fc6b2428b981acd6c3c238b5f (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:44.605595   996 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:44.613132   854 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.614938   929 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.615757  1871 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:44.617302   968 consensus_peers.cc:597] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d5a097d0fe49bab1a3ec1f78f4164d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.618363  1835 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.628086  1871 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.628556   854 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.640098   924 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.648355   929 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:44.664271   751 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.667645   846 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:44.668118   894 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01fc410be7f74515adce264a85731b90. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.669080  1873 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.673202   996 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.688697  1872 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:44.690112   876 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7980 after lost signal to thread 32227
W20260709 13:12:44.691588   876 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7820 after lost signal to thread 787
W20260709 13:12:44.693166   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800330860 after lost signal to thread 879
W20260709 13:12:44.694535   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800330e80 after lost signal to thread 954
W20260709 13:12:44.696677   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800135ba0 after lost signal to thread 1028
W20260709 13:12:44.698206   876 debug-util.cc:398] Leaking SignalData structure 0x7b080009f5a0 after lost signal to thread 1625
I20260709 13:12:44.707226   846 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.720801   996 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:44.723768  1878 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.740013   750 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:44.762864   998 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:44.785180   968 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6444551c84374ec98f4becf8b07446ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.789095   846 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.789211   929 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.790743  1835 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.791522  1870 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.815654  1835 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.817870   846 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.820564   929 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.835685   750 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.839062   846 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:12:44.849749   819 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e65c03394bf04e2fb03562b27d85f5b3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.853006   996 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.854604   929 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.855964  1840 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.857364  1842 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.885133  1842 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.885710   929 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.908495   998 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.911957   751 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:44.995205  1899 ts_tablet_manager.cc:933] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:45.008863  1899 tablet_copy_client.cc:323] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:45.010813  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet d3d5a097d0fe49bab1a3ec1f78f4164d from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:45.011351  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet d3d5a097d0fe49bab1a3ec1f78f4164d from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-d3d5a097d0fe49bab1a3ec1f78f4164d
I20260709 13:12:45.019116  1014 tablet_copy_source_session.cc:215] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.023836  1899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d5a097d0fe49bab1a3ec1f78f4164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.061082  1899 tablet_copy_client.cc:806] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.061683  1899 tablet_copy_client.cc:670] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.092015  1899 tablet_copy_client.cc:538] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.092574  1901 ts_tablet_manager.cc:933] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:45.105198  1899 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:45.113631  1901 tablet_copy_client.cc:323] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:45.115851   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 01fc410be7f74515adce264a85731b90 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:45.116385   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 01fc410be7f74515adce264a85731b90 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-01fc410be7f74515adce264a85731b90
I20260709 13:12:45.127856   940 tablet_copy_source_session.cc:215] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.131433  1901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01fc410be7f74515adce264a85731b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.174299  1904 ts_tablet_manager.cc:933] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:45.175196  1901 tablet_copy_client.cc:806] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.175813  1901 tablet_copy_client.cc:670] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.179895  1901 tablet_copy_client.cc:538] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.184986  1899 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.188319  1899 tablet_bootstrap.cc:492] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:45.189143  1899 ts_tablet_manager.cc:1403] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.084s	user 0.026s	sys 0.004s
I20260709 13:12:45.196650  1899 raft_consensus.cc:359] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.189457  1904 tablet_copy_client.cc:323] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:45.199203  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 6444551c84374ec98f4becf8b07446ba from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:45.199792  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 6444551c84374ec98f4becf8b07446ba from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-6444551c84374ec98f4becf8b07446ba
I20260709 13:12:45.201139  1901 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:45.214147  1014 tablet_copy_source_session.cc:215] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.197652  1899 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:45.221360  1899 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.222057  1904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6444551c84374ec98f4becf8b07446ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.264119  1899 ts_tablet_manager.cc:1434] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.027s
I20260709 13:12:45.268599  1904 tablet_copy_client.cc:806] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.269178  1904 tablet_copy_client.cc:670] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.269623  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-d3d5a097d0fe49bab1a3ec1f78f4164d received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:45.270155  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-d3d5a097d0fe49bab1a3ec1f78f4164d on tablet d3d5a097d0fe49bab1a3ec1f78f4164d with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:45.277647  1904 tablet_copy_client.cc:538] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.286753  1904 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:45.289602  1901 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.290627  1901 tablet_bootstrap.cc:492] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:45.291321  1901 ts_tablet_manager.cc:1403] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.090s	user 0.017s	sys 0.013s
I20260709 13:12:45.300997  1901 raft_consensus.cc:359] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.301975  1901 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:45.302646  1901 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.305346  1901 ts_tablet_manager.cc:1434] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 13:12:45.307114   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-01fc410be7f74515adce264a85731b90 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:45.307570   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-01fc410be7f74515adce264a85731b90 on tablet 01fc410be7f74515adce264a85731b90 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:45.317827  1904 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.318910  1904 tablet_bootstrap.cc:492] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:45.319636  1904 ts_tablet_manager.cc:1403] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.004s
I20260709 13:12:45.322628  1904 raft_consensus.cc:359] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.323446  1904 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:45.324123  1904 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.326773  1904 ts_tablet_manager.cc:1434] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:45.332479  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-6444551c84374ec98f4becf8b07446ba received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:45.332927  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-6444551c84374ec98f4becf8b07446ba on tablet 6444551c84374ec98f4becf8b07446ba with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:45.425267  1904 ts_tablet_manager.cc:933] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:45.447072  1904 tablet_copy_client.cc:323] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:12:45.448698   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet e65c03394bf04e2fb03562b27d85f5b3 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:12:45.449222   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet e65c03394bf04e2fb03562b27d85f5b3 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-e65c03394bf04e2fb03562b27d85f5b3
I20260709 13:12:45.455426   865 tablet_copy_source_session.cc:215] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.472107  1904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65c03394bf04e2fb03562b27d85f5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.497800  1904 tablet_copy_client.cc:806] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.498395  1904 tablet_copy_client.cc:670] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.513080  1904 tablet_copy_client.cc:538] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.526340  1904 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:45.554595  1904 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.555540  1904 tablet_bootstrap.cc:492] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:45.556246  1904 ts_tablet_manager.cc:1403] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.030s	user 0.029s	sys 0.000s
I20260709 13:12:45.566470  1904 raft_consensus.cc:359] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.567467  1904 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:45.568300  1904 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:45.570925  1904 ts_tablet_manager.cc:1434] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:12:45.572909   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-e65c03394bf04e2fb03562b27d85f5b3 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:12:45.573325   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-e65c03394bf04e2fb03562b27d85f5b3 on tablet e65c03394bf04e2fb03562b27d85f5b3 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:45.588495  1601 raft_consensus.cc:1217] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:45.708833  1601 raft_consensus.cc:1217] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:45.737895  1600 raft_consensus.cc:1217] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:45.922432  1601 raft_consensus.cc:1217] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:46.028654  1913 raft_consensus.cc:1064] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:46.031859  1913 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.041781  1601 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.046301  1911 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.049423   927 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.050403   853 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.051817  1913 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.055083  1870 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.079787  1870 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.083133  1601 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.101925   753 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.101926   853 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.140365   927 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.177364  1870 raft_consensus.cc:1064] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:46.180171  1870 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.191319  1601 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.192646   929 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.193264  1871 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.193931   853 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.196060  1835 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.196897  1871 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.223977  1911 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.228683   853 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.247444   929 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.251863  1001 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.281325  1601 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.282015   751 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.282862  1870 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.285039   929 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.287395  1870 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.298321  1601 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.307276  1600 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.299031  1911 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.312314  1878 raft_consensus.cc:1064] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:46.315050  1878 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.325101   996 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.336746   927 raft_consensus.cc:2955] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.345584   750 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.350273   853 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.351778  1870 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.362422   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d3d5a097d0fe49bab1a3ec1f78f4164d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.364192   854 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.376994  1870 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.398550   843 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d5a097d0fe49bab1a3ec1f78f4164d with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:46.415583  1600 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.416625  1001 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.424423  1878 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.431938   854 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.433175  1873 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:46.426555  1598 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.436508  1840 raft_consensus.cc:1064] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:46.441392  1872 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:12:46.441946  1931 tablet_replica.cc:333] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:46.450240  1911 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:12:46.439260  1840 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.503474  1931 raft_consensus.cc:2243] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.516312   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6444551c84374ec98f4becf8b07446ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.518141   750 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.539994  1931 raft_consensus.cc:2272] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.540759  1601 raft_consensus.cc:2955] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:46.612495   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 81908.9us
W20260709 13:12:46.616488   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 72753.7us
W20260709 13:12:46.651814   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 146137us
I20260709 13:12:46.652089   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 6444551c84374ec98f4becf8b07446ba with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:46.658571   854 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.674538  1598 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.674220  1931 ts_tablet_manager.cc:1916] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.690498  1878 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.697029   924 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.707362  1001 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.711885  1601 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.729511  1897 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.719141   750 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.734090  1935 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:46.743659   996 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.749969  1944 tablet_replica.cc:333] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:46.757566   927 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.762223  1944 raft_consensus.cc:2243] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.769296  1944 raft_consensus.cc:2272] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.764678  1598 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.785913  1937 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:46.788352  1944 ts_tablet_manager.cc:1916] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.794027  1878 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.796216  1598 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.778513  1931 ts_tablet_manager.cc:1929] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:46.783810   854 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:12:46.805423  1945 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:12:46.805590  1931 log.cc:1199] T d3d5a097d0fe49bab1a3ec1f78f4164d P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/d3d5a097d0fe49bab1a3ec1f78f4164d
I20260709 13:12:46.806571  1935 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:12:46.806464  1928 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:12:46.812057   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 01fc410be7f74515adce264a85731b90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.827476   750 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.840440   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet d3d5a097d0fe49bab1a3ec1f78f4164d (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:46.849889   846 raft_consensus.cc:2955] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.852926  1944 ts_tablet_manager.cc:1929] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:46.889570  1944 log.cc:1199] T 6444551c84374ec98f4becf8b07446ba P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/6444551c84374ec98f4becf8b07446ba
I20260709 13:12:46.888630  1842 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.889830  1601 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.904042   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 01fc410be7f74515adce264a85731b90 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:46.916599   750 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.923192   928 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.926580   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 6444551c84374ec98f4becf8b07446ba (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:46.936420  1948 tablet_replica.cc:333] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:46.948215  1001 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:46.955410  1948 raft_consensus.cc:2243] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.956419  1948 raft_consensus.cc:2272] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.962026   846 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:46.967428  1948 ts_tablet_manager.cc:1916] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.977653   929 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.980196  1897 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.982496  1601 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.984102  1897 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.999455  1915 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:47.017627  1601 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:47.026108   927 raft_consensus.cc:2955] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:47.032037   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e65c03394bf04e2fb03562b27d85f5b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:47.048693   751 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.072039   994 tablet_service.cc:1558] Processing DeleteTablet for tablet e65c03394bf04e2fb03562b27d85f5b3 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:47.075009  1948 ts_tablet_manager.cc:1929] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:47.085572  1948 log.cc:1199] T 01fc410be7f74515adce264a85731b90 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/01fc410be7f74515adce264a85731b90
I20260709 13:12:47.090063  1948 tablet_replica.cc:333] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:47.090315   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 01fc410be7f74515adce264a85731b90 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:47.091753  1948 raft_consensus.cc:2243] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:47.092633  1948 raft_consensus.cc:2272] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.095815  1948 ts_tablet_manager.cc:1916] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:47.113235  1948 ts_tablet_manager.cc:1929] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:47.113780  1948 log.cc:1199] T e65c03394bf04e2fb03562b27d85f5b3 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/e65c03394bf04e2fb03562b27d85f5b3
I20260709 13:12:47.115933   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet e65c03394bf04e2fb03562b27d85f5b3 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:48.146180   995 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.155525   928 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.156760   854 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.156807   968 consensus_peers.cc:597] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14a8781662d54452a1478c2ae0e3f911. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.157887  1949 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.158771  1950 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.180246  1950 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.202086   854 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.208164   751 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.209277   927 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.214502   995 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:12:48.239952   968 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53d4a96b7f094cf691b31dbe6fb015bd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.240916   927 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.243376  1943 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:48.241290   846 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.245997  1949 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.275990  1870 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.281895   854 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.308868   928 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.315301   749 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.324970   929 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:12:48.332984   894 consensus_peers.cc:597] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b871863269841f7b0aa8baf5e640865. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.343566  1001 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.345682  1951 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.350193   854 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.351747  1951 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:48.355247  1878 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.361294  1001 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.368957   927 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.374177   749 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.392118  1001 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.392861   894 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 633fa6f0c6434c49aeb87667c9d97141. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.393530  1878 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.398490   846 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.402904   854 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.405938  1937 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.458738  1937 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.463284   853 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.467319  1001 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:48.494496   749 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.736625  1969 ts_tablet_manager.cc:933] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:48.742719  1969 tablet_copy_client.cc:323] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:48.744982  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 14a8781662d54452a1478c2ae0e3f911 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:48.745565  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 14a8781662d54452a1478c2ae0e3f911 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-14a8781662d54452a1478c2ae0e3f911
I20260709 13:12:48.756053  1014 tablet_copy_source_session.cc:215] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.761246  1969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a8781662d54452a1478c2ae0e3f911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.781988  1969 tablet_copy_client.cc:806] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.782713  1969 tablet_copy_client.cc:670] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.784186  1972 ts_tablet_manager.cc:933] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:48.804229  1969 tablet_copy_client.cc:538] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.804387  1972 tablet_copy_client.cc:323] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:48.806483   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 633fa6f0c6434c49aeb87667c9d97141 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:48.807042   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 633fa6f0c6434c49aeb87667c9d97141 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-633fa6f0c6434c49aeb87667c9d97141
I20260709 13:12:48.817830   940 tablet_copy_source_session.cc:215] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.821260  1972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 633fa6f0c6434c49aeb87667c9d97141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.837802  1969 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:48.847918  1972 tablet_copy_client.cc:806] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.862146  1972 tablet_copy_client.cc:670] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.866775  1972 tablet_copy_client.cc:538] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.879822  1972 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:48.905864  1974 ts_tablet_manager.cc:933] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:48.908900  1969 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.952262  1974 tablet_copy_client.cc:323] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:48.954238  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 53d4a96b7f094cf691b31dbe6fb015bd from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:48.954808  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 53d4a96b7f094cf691b31dbe6fb015bd from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-53d4a96b7f094cf691b31dbe6fb015bd
I20260709 13:12:48.964864  1014 tablet_copy_source_session.cc:215] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.966717  1969 tablet_bootstrap.cc:492] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:48.967588  1969 ts_tablet_manager.cc:1403] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.130s	user 0.030s	sys 0.010s
I20260709 13:12:48.973135  1978 ts_tablet_manager.cc:933] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:48.974326  1974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d4a96b7f094cf691b31dbe6fb015bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.974753  1972 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.977142  1972 tablet_bootstrap.cc:492] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:48.986263  1972 ts_tablet_manager.cc:1403] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.107s	user 0.022s	sys 0.018s
I20260709 13:12:48.989770  1972 raft_consensus.cc:359] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.990715  1972 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:48.971055  1969 raft_consensus.cc:359] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.992077  1969 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:48.992792  1969 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:48.994704  1978 tablet_copy_client.cc:323] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:49.000928   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 3b871863269841f7b0aa8baf5e640865 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:49.001482   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 3b871863269841f7b0aa8baf5e640865 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-3b871863269841f7b0aa8baf5e640865
I20260709 13:12:48.991490  1972 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:49.006529  1972 ts_tablet_manager.cc:1434] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:12:49.008564   939 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-633fa6f0c6434c49aeb87667c9d97141 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:49.009060   939 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-633fa6f0c6434c49aeb87667c9d97141 on tablet 633fa6f0c6434c49aeb87667c9d97141 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:49.034616  1969 ts_tablet_manager.cc:1434] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:12:49.036821  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-14a8781662d54452a1478c2ae0e3f911 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:49.037328  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-14a8781662d54452a1478c2ae0e3f911 on tablet 14a8781662d54452a1478c2ae0e3f911 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:49.051932   940 tablet_copy_source_session.cc:215] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:49.056474  1978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b871863269841f7b0aa8baf5e640865. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:49.081231   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52516.2us
I20260709 13:12:49.081475  1974 tablet_copy_client.cc:806] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:49.084036  1974 tablet_copy_client.cc:670] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:49.088091  1974 tablet_copy_client.cc:538] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:49.101672  1974 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:49.112267  1978 tablet_copy_client.cc:806] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:49.113022  1978 tablet_copy_client.cc:670] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:49.117100  1978 tablet_copy_client.cc:538] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:49.131647  1978 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:49.189350  1978 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:49.189628  1974 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:49.194305  1974 tablet_bootstrap.cc:492] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:49.195154  1974 ts_tablet_manager.cc:1403] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.094s	user 0.037s	sys 0.000s
I20260709 13:12:49.190425  1978 tablet_bootstrap.cc:492] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:49.196247  1978 ts_tablet_manager.cc:1403] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.065s	user 0.036s	sys 0.016s
I20260709 13:12:49.199702  1978 raft_consensus.cc:359] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:49.200677  1978 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:49.201478  1978 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:49.202828  1974 raft_consensus.cc:359] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:49.203897  1974 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:49.204604  1974 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:49.203487  1978 ts_tablet_manager.cc:1434] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:49.206835  1974 ts_tablet_manager.cc:1434] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:12:49.208588   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-3b871863269841f7b0aa8baf5e640865 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:49.209026   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-3b871863269841f7b0aa8baf5e640865 on tablet 3b871863269841f7b0aa8baf5e640865 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:49.212976  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-53d4a96b7f094cf691b31dbe6fb015bd received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:49.213440  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-53d4a96b7f094cf691b31dbe6fb015bd on tablet 53d4a96b7f094cf691b31dbe6fb015bd with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:49.325256  1601 raft_consensus.cc:1217] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.353479  1598 raft_consensus.cc:1217] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.361472  1598 raft_consensus.cc:1217] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.348309  1601 raft_consensus.cc:1217] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.878094  1870 raft_consensus.cc:1064] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:49.879917  1870 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:49.886109  1601 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.887579  1950 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.888135   846 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.889504   927 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.890451  1950 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.893666  1870 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:49.908608  1950 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:49.912523  1601 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:49.933220  1981 raft_consensus.cc:1064] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:49.944964   927 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:49.950081  1986 raft_consensus.cc:1064] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:49.954393   751 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.956245   853 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:49.955371   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.012332  1986 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:49.999730  1981 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.033233   929 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.035000  1601 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.037248  1870 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.038851  1870 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.045944   846 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.047488  1870 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.048789  1951 raft_consensus.cc:1064] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:50.051187  1951 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.110608  1599 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.114048  1993 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.119645  1001 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.122670  1937 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:50.150739  1599 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.154127  1981 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.155244   850 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.156526   854 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.191044  1994 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.190886  1870 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.193015  1994 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.195619  1598 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.205252   995 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:50.204396   929 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.211091  1981 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.213110   850 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.216632  1937 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:50.222879  1599 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.227010   995 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.239009   995 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.260818   851 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.260270  1951 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.266829  1599 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.274206  1950 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:50.279690  1001 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.287629  1949 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.292316   853 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.270519   752 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.295156  1599 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.317780   752 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:50.337767   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 100535us
I20260709 13:12:50.341795  1870 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.353641   846 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.357654   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53d4a96b7f094cf691b31dbe6fb015bd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.360488   750 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.362314   995 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.354943  1599 raft_consensus.cc:2955] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.374245   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 53d4a96b7f094cf691b31dbe6fb015bd with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:12:50.354652   854 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.380035   929 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.381830  1870 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.390630  1600 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.394659  2013 tablet_replica.cc:333] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:12:50.395841  2013 raft_consensus.cc:2243] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.396687  2013 raft_consensus.cc:2272] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.400622  2013 ts_tablet_manager.cc:1916] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.399505  1986 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.386822   750 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.406905   929 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.409597  1870 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.417788  1599 raft_consensus.cc:2955] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.420054   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 14a8781662d54452a1478c2ae0e3f911 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.434283   751 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:50.445778   929 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.452358   927 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:50.453214  1599 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.443748  2013 ts_tablet_manager.cc:1929] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.454690  2013 log.cc:1199] T 53d4a96b7f094cf691b31dbe6fb015bd P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/53d4a96b7f094cf691b31dbe6fb015bd
I20260709 13:12:50.455065   846 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.456367   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 53d4a96b7f094cf691b31dbe6fb015bd (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:50.456533  1981 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.457676  1993 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.465377  1599 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.466130   995 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.467775  1951 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.470748  2003 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:50.479193   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 14a8781662d54452a1478c2ae0e3f911 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:50.480496  1994 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.484310  1600 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.503849   995 raft_consensus.cc:2955] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.508633  2014 tablet_replica.cc:333] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:50.511698   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3b871863269841f7b0aa8baf5e640865 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.512409  1937 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.515837  1598 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.513597   753 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.521181   854 raft_consensus.cc:2955] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.552412   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 633fa6f0c6434c49aeb87667c9d97141 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.553265   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b871863269841f7b0aa8baf5e640865 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:12:50.554229  2014 raft_consensus.cc:2243] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.554883  2014 raft_consensus.cc:2272] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.559454  2014 ts_tablet_manager.cc:1916] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.559942   753 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:50.579310   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 633fa6f0c6434c49aeb87667c9d97141 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:12:50.586730  2016 tablet_replica.cc:333] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:12:50.587997  2016 raft_consensus.cc:2243] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.588944  2016 raft_consensus.cc:2272] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.589008  2014 ts_tablet_manager.cc:1929] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.590709  2014 log.cc:1199] T 14a8781662d54452a1478c2ae0e3f911 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/14a8781662d54452a1478c2ae0e3f911
I20260709 13:12:50.592662   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 14a8781662d54452a1478c2ae0e3f911 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:50.592703  2014 tablet_replica.cc:333] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:12:50.592828  2016 ts_tablet_manager.cc:1916] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.594015  2014 raft_consensus.cc:2243] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.594667  2014 raft_consensus.cc:2272] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.598881  2014 ts_tablet_manager.cc:1916] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.609476  2016 ts_tablet_manager.cc:1929] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.610073  2016 log.cc:1199] T 633fa6f0c6434c49aeb87667c9d97141 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/633fa6f0c6434c49aeb87667c9d97141
I20260709 13:12:50.611765   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 633fa6f0c6434c49aeb87667c9d97141 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:50.611899  2014 ts_tablet_manager.cc:1929] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.612371  2014 log.cc:1199] T 3b871863269841f7b0aa8baf5e640865 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/3b871863269841f7b0aa8baf5e640865
I20260709 13:12:50.613736   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 3b871863269841f7b0aa8baf5e640865 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:12:51.676451   997 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:12:51.682199   968 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90a3708334cc4bd59c73d494d2747d54. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.683705   854 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.685382  1949 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.686257   927 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.687395  1949 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.692808  2020 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.694550   854 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.696623   927 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.708943   750 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.709189   997 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:51.726312   927 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.726636   854 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:51.726720   968 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7cf198f3795b43ff951a11167bc21580. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.727742  2020 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.730566  1949 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.739629  1949 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.742861   854 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.741269   927 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:51.752941   929 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:51.762415   997 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.764142  2017 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:51.766554   894 consensus_peers.cc:597] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8c347b6586f486192f48c46974a2526. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.762825   854 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.767784   753 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.768775  1994 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:52.432436   854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.017s	sys 0.005s
W20260709 13:12:52.432830   854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.017s	sys 0.005s
W20260709 13:12:52.612267   896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.004s	sys 0.025s
W20260709 13:12:52.612747   896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.004s	sys 0.025s
W20260709 13:12:52.613009   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 834835us
I20260709 13:12:52.627197  2051 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:52.668116  2056 ts_tablet_manager.cc:933] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:52.673225  2058 ts_tablet_manager.cc:933] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:52.674296  2055 ts_tablet_manager.cc:933] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:52.677037  2056 tablet_copy_client.cc:323] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:52.679309  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 90a3708334cc4bd59c73d494d2747d54 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:52.679958  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 90a3708334cc4bd59c73d494d2747d54 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-90a3708334cc4bd59c73d494d2747d54
I20260709 13:12:52.681061  2055 tablet_copy_client.cc:323] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:12:52.684365   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet b8c347b6586f486192f48c46974a2526 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:12:52.684890   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet b8c347b6586f486192f48c46974a2526 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-b8c347b6586f486192f48c46974a2526
I20260709 13:12:52.693055  1014 tablet_copy_source_session.cc:215] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:52.696741  2058 tablet_copy_client.cc:323] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:12:52.700810  2061 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:52.701414  2061 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:52.719532   940 tablet_copy_source_session.cc:215] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:52.722596   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:52.724920  2061 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:52.744247   995 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:52.414567 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:52.758669   850 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:52.770924   750 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.774979  2055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c347b6586f486192f48c46974a2526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:52.776439  2056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a3708334cc4bd59c73d494d2747d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:52.819514   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:52.825011  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 7cf198f3795b43ff951a11167bc21580 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:12:52.825289  1567 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:52.825654  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 7cf198f3795b43ff951a11167bc21580 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-7cf198f3795b43ff951a11167bc21580
W20260709 13:12:52.833937   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:52.836517   819 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7de58b0d5ad94d869fac3b0491143543. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:52.843101  2061 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:52.856323  1014 tablet_copy_source_session.cc:215] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:52.863951  2056 tablet_copy_client.cc:806] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:52.864598  2056 tablet_copy_client.cc:670] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:52.867864  2058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf198f3795b43ff951a11167bc21580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:52.879768  2056 tablet_copy_client.cc:538] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:52.881567   855 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:52.885070  2036 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:52.885586  2036 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:52.888168  2036 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:52.888602   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 145231us
I20260709 13:12:52.889436   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:52.891235   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:52.893065   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:52.894068  2036 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:52.902526  2055 tablet_copy_client.cc:806] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:52.903169  2055 tablet_copy_client.cc:670] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:52.906302   928 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:52.908102  2055 tablet_copy_client.cc:538] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:52.922634  2056 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:52.927774  2041 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.948125   997 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:52.948571   995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:52.959178   995 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:52.973356  2037 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:52.986934  2055 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:53.007234   997 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:53.009281   997 raft_consensus.cc:606] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:53.019338   997 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:53.019928   997 raft_consensus.cc:606] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
W20260709 13:12:53.036723  2069 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:12:53.037278   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 69102.2us
I20260709 13:12:53.091351  2041 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:53.093087   929 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:53.101164  2056 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:53.101960  2058 tablet_copy_client.cc:806] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:53.104378  2058 tablet_copy_client.cc:670] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:53.102598  2056 tablet_bootstrap.cc:492] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:53.106413  2056 ts_tablet_manager.cc:1403] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.184s	user 0.034s	sys 0.005s
I20260709 13:12:53.107877   997 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:53.110461  2058 tablet_copy_client.cc:538] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:53.117904  2056 raft_consensus.cc:359] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.119450   749 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 2, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:12:53.124677   929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:12:53.128777  2056 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:53.129061   929 raft_consensus.cc:606] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:12:53.129727  2056 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.132462  2056 ts_tablet_manager.cc:1434] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:12:53.137476  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-90a3708334cc4bd59c73d494d2747d54 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:53.137924  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-90a3708334cc4bd59c73d494d2747d54 on tablet 90a3708334cc4bd59c73d494d2747d54 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:53.149703  2055 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:53.150734  2055 tablet_bootstrap.cc:492] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:53.151469  2055 ts_tablet_manager.cc:1403] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.165s	user 0.024s	sys 0.007s
I20260709 13:12:53.170195  2055 raft_consensus.cc:359] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.170868  2055 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:53.171409  2055 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.173352  2055 ts_tablet_manager.cc:1434] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:12:53.192282  2058 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:53.197886   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-b8c347b6586f486192f48c46974a2526 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:12:53.198388   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-b8c347b6586f486192f48c46974a2526 on tablet b8c347b6586f486192f48c46974a2526 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:53.222164  2054 raft_consensus.cc:993] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:12:53.222807  2049 raft_consensus.cc:1081] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:12:53.228686   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:12:53.229283   850 raft_consensus.cc:606] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:53.229636   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51144
I20260709 13:12:53.230280   997 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.230634   997 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.233547  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:53.234210  1002 raft_consensus.cc:606] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:53.238973  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:53.239804  1002 raft_consensus.cc:606] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:53.241374 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:12:53.241074   997 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:53.246064   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:53.246381   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:53.246676   930 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.247080 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:53.246981   930 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:53.247963   930 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:53.248993   930 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.246959   850 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.251756  2046 raft_consensus.cc:993] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.252507  2046 raft_consensus.cc:1081] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.243652   997 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:53.253983  1599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:45838
I20260709 13:12:53.254482  1599 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.254815  1599 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.259733   930 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:53.261350  1599 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.264078   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:53.264961  2072 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.265355  1599 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:53.262786   850 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:53.266577   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:53.266703  2072 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.267158   846 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.267503   846 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:53.268035  2072 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:53.270062   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:53.270603   929 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.268224   846 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.272239   846 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.285480   929 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.287045   846 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.288282  1566 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:53.289170  2061 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.292166  2061 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:53.293094  2061 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.297108  2058 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:53.298190  2058 tablet_bootstrap.cc:492] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:12:53.306107  2055 ts_tablet_manager.cc:933] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:53.306646   749 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:53.310875  2058 ts_tablet_manager.cc:1403] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.017s
I20260709 13:12:53.316464   750 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:53.323220  2058 raft_consensus.cc:359] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.325019  2058 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:53.325685  2058 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:53.328099  2058 ts_tablet_manager.cc:1434] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:53.330950  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-7cf198f3795b43ff951a11167bc21580 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:12:53.331369  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-7cf198f3795b43ff951a11167bc21580 on tablet 7cf198f3795b43ff951a11167bc21580 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:53.334563  2055 tablet_copy_client.cc:323] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:12:53.336867   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet 7de58b0d5ad94d869fac3b0491143543 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:12:53.337318   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet 7de58b0d5ad94d869fac3b0491143543 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-7de58b0d5ad94d869fac3b0491143543
I20260709 13:12:53.351477   865 tablet_copy_source_session.cc:215] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:53.355582  2055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de58b0d5ad94d869fac3b0491143543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:53.368614  2055 tablet_copy_client.cc:806] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:53.369227  2055 tablet_copy_client.cc:670] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:53.374850  2055 tablet_copy_client.cc:538] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:53.385093  2055 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:12:53.389875  2040 raft_consensus.cc:993] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:12:53.390601  2035 raft_consensus.cc:1081] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:12:53.392597   846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:12:53.393177   846 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.393532   846 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.400257  2040 raft_consensus.cc:993] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.400888  2071 raft_consensus.cc:1081] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.407191  1599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:12:53.407977  1599 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.408425  1599 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.415623  2040 raft_consensus.cc:993] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.416159  2040 raft_consensus.cc:1081] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
W20260709 13:12:53.426482  1025 debug-util.cc:398] Leaking SignalData structure 0x7b0800082bc0 after lost signal to thread 32227
W20260709 13:12:53.427666  1025 debug-util.cc:398] Leaking SignalData structure 0x7b0800037280 after lost signal to thread 787
W20260709 13:12:53.428898  1025 debug-util.cc:398] Leaking SignalData structure 0x7b08002befa0 after lost signal to thread 879
W20260709 13:12:53.430006  1025 debug-util.cc:398] Leaking SignalData structure 0x7b0800046500 after lost signal to thread 954
I20260709 13:12:53.430151  1597 raft_consensus.cc:1217] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:12:53.440068  1025 debug-util.cc:398] Leaking SignalData structure 0x7b080026f9e0 after lost signal to thread 1028
W20260709 13:12:53.442662  1025 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad340 after lost signal to thread 1625
I20260709 13:12:53.465559  1593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:12:53.466570  1593 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.467132  1593 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.526824  2071 raft_consensus.cc:993] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.528075  2042 raft_consensus.cc:1081] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.528789  2089 raft_consensus.cc:993] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.529259  2089 raft_consensus.cc:1081] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:53.533836  1600 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:12:53.534693  1600 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.535167  1600 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.536335  1597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:12:53.537034  1597 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:53.537479  1597 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.667155  1592 raft_consensus.cc:1217] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:53.694053  1592 raft_consensus.cc:1217] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:53.731715   846 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:53.735392   846 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:53.761371   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:53.761986   997 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.762272   997 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.763000   997 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:53.764045   997 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.766315  1600 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.768266  1599 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.771003   997 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:12:53.773943  1599 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:53.775348  1593 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.776460  1592 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:53.778261  2088 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:53.779090  1593 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:53.779263  2088 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.779656  2088 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.780067   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:53.780301   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:53.780776   855 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.780972   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:53.781491   930 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.785209   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:53.785763   999 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.786177   999 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.785687  2088 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.790904  2088 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.791039   999 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.792393   999 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.796905   855 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.798054  1566 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:53.798938  2061 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.799139   999 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.821112  1597 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.825268   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:53.826982   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 331655us
I20260709 13:12:53.853596   929 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.860769   929 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.861595  2087 raft_consensus.cc:1064] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:53.863994   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:53.864641   850 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.864529  2087 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.830294   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:53.845594   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:53.875523   999 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.875891   999 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.876590   999 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.877454   999 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:53.883280   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.885926   999 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.853068   855 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:53.886713   846 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.891060   850 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.892202  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:53.892199  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:12:53.889779  1599 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:53.852859  1600 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:53.908877   846 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:53.909263  2093 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.910346   928 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:53.911988  2094 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.914090  2094 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:53.914630  2093 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:53.916204  2093 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.917454  2094 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:12:53.932029   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 83950.5us
I20260709 13:12:53.795253   930 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:12:53.943550   928 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
W20260709 13:12:53.951511   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58903us
I20260709 13:12:53.958633  1599 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.958673  2072 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:12:53.961632   928 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:53.964695   846 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.940091  2061 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:53.968386  2072 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.954546  2088 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:12:53.976367  2088 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:53.976739  2088 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:53.977497  2088 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.978950  2088 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.980414  2089 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:53.992947   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:54.009526   997 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
W20260709 13:12:54.010402  2055 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.118s	sys 0.212s
I20260709 13:12:53.974627  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:12:54.026826  2055 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.118s	sys 0.213s
W20260709 13:12:54.027058  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52530.7us
I20260709 13:12:54.028393  2097 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.029040  2097 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:54.030103  2097 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:53.991472  1597 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:54.011665  2061 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:54.021741   999 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:54.046100  2092 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:12:54.059623   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 96080.8us
W20260709 13:12:54.064905   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 44274.3us
I20260709 13:12:54.070211   923 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:54.071714   924 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:54.089202   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:54.091403  2101 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.092015  2101 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:54.093071  2101 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:54.101941  2055 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:12:54.103255   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:54.191325   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 95398.3us
W20260709 13:12:54.190984   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 120158us
I20260709 13:12:54.165915  2055 tablet_bootstrap.cc:492] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
W20260709 13:12:54.240737   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 179481us
I20260709 13:12:54.242668  2055 ts_tablet_manager.cc:1403] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.858s	user 0.136s	sys 0.222s
I20260709 13:12:54.259349 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:54.288328   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46798.2us
I20260709 13:12:54.331208   855 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:12:54.337857   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 144956us
I20260709 13:12:54.357039  2092 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:54.373097  2052 raft_consensus.cc:1064] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:54.376595  2052 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:54.382452  2055 raft_consensus.cc:359] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:54.383591  2055 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:12:54.408094   999 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:54.414311  2061 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:54.418959  2049 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:54.419629  2049 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:54.443369  2049 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:54.427863  2055 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:54.447485  2055 ts_tablet_manager.cc:1434] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.185s	user 0.010s	sys 0.001s
I20260709 13:12:54.452435   750 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.462334   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:54.473385   930 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:54.480440   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:54.500617   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 423665us
I20260709 13:12:54.506880   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-7de58b0d5ad94d869fac3b0491143543 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:12:54.507398   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-7de58b0d5ad94d869fac3b0491143543 on tablet 7de58b0d5ad94d869fac3b0491143543 with peer 597687e464174d1ea6043161e037d240
I20260709 13:12:54.508235   999 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:54.484972  2061 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:12:54.511855  2090 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:12:54.566646   895 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:54.579573  2111 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.556931  2089 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:12:54.605751  1001 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:54.561553  2103 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:12:54.707419   846 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:54.670145  2097 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:12:54.793941 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:54.820514  1599 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:12:54.817093  2121 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 13:12:54.829336  2117 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:12:54.829195  2049 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:54.929558  2139 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:54.945163   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 238597us
I20260709 13:12:54.949342   753 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:54.952526  2118 raft_consensus.cc:1064] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:54.952339  1600 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:54.955258  1002 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:54.955004  2118 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:12:54.960093   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 189055us
I20260709 13:12:54.975591  2106 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:12:54.981354   997 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:12:55.095036   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 73125.8us
W20260709 13:12:55.095263   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 111446us
W20260709 13:12:55.101421  2134 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:12:55.136005   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 43699.4us
W20260709 13:12:55.139849   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 112093us
W20260709 13:12:55.144130   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 43570.4us
W20260709 13:12:55.155629   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46816.6us
I20260709 13:12:55.171815   855 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:55.174904  1600 raft_consensus.cc:1217] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:55.146291  2092 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:12:55.201810  2144 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 13:12:55.202306   848 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:12:55.255769   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 119028us
W20260709 13:12:55.271296   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 170156us
I20260709 13:12:55.281574   850 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:55.310060  2089 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:55.339337  2061 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 13:12:55.369417   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57045.5us
W20260709 13:12:55.362000  2151 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:12:55.337152  2089 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.349242  2097 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:12:55.350193  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 153653us
I20260709 13:12:55.396638  2150 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.400300   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:55.402819  2153 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:55.404392  2153 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:55.397190  2150 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.397699  2089 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:55.406164  1593 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:55.425969  2150 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:55.430352   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:55.429139  2049 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:55.437355  2049 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.450708   855 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:55.543085   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:55.438133  2150 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
W20260709 13:12:55.508276  2162 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:12:55.545529  2153 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:55.622295  2144 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:12:55.647687  2160 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:12:55.629406  2052 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:55.653178  2051 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:55.653832  2051 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:55.655490  2054 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.653662  2052 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.656032  2054 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.656342  2049 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:55.664891  2051 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:55.677182  2085 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:12:55.682646  2129 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:55.683450  2052 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:55.691499   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 383234us
I20260709 13:12:55.702973   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:55.683422  2129 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:55.718160  2054 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:55.741083  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:55.740677  2154 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:55.879034  2129 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:55.922338   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 295906us
W20260709 13:12:55.935509  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 76437.5us
W20260709 13:12:55.882931  2090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.018s	sys 0.024s
W20260709 13:12:55.938194  2090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.019s	sys 0.025s
W20260709 13:12:55.941310   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 268120us
I20260709 13:12:55.942384  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:55.973021  1001 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:12:56.005013   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 55487.5us
I20260709 13:12:56.026883  2179 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.042922  2099 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:56.055089  2153 raft_consensus.cc:1064] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:12:56.053050  2099 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:56.028169  2179 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:56.063815   850 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:56.066664  2099 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.096030  2150 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:56.096242   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:56.096939  2150 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:56.120327  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:56.121181  2179 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.104431  2104 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:12:56.094252  2153 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.131991  2104 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:56.145709   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.133947  2150 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:56.149012   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 89747.5us
W20260709 13:12:56.191681  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 43707.8us
I20260709 13:12:56.195052   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:56.200644   999 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:56.205554  2160 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:56.223657  2154 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:56.224341  2154 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:56.195160   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:56.132946  2104 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:56.228945   995 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:12:56.230062  2120 raft_consensus.cc:1027] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: attempt to promote peer 597687e464174d1ea6043161e037d240: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 13:12:56.291276   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60677.4us
W20260709 13:12:56.291857   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64837.2us
I20260709 13:12:56.225440  2095 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:56.230814  2154 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.206408  2160 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.308182  2104 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:56.311596  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73852.8us
I20260709 13:12:56.293133  1593 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:56.367419  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 174705us
I20260709 13:12:56.312676  2095 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:56.394053  2188 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 13:12:56.425133  2095 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.425983  1594 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:56.442689  2054 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.443625  2054 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.447750   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 110) took 1971 ms. Trace:
I20260709 13:12:56.447934  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:56.458274   753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 218) took 1538 ms. Trace:
I20260709 13:12:56.461211  2085 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.458819   753 rpcz_store.cc:276] 0709 13:12:54.919330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.919476 (+   146us) service_pool.cc:224] Handling call
0709 13:12:54.934702 (+ 15226us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02b47f3a 00007f0d088fe1cc 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc 00007f0d06ebc392 00007f0d00e7a48c 00007f0d00e798c6
0709 13:12:56.458065 (+1523363us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:55.425698 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:55.426268 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:55.426318 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:12:55.426329 (+    11us) tablet.cc:662] Decoding operations
0709 13:12:55.426640 (+   311us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:55.426689 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:55.426715 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:55.426919 (+   204us) tablet.cc:701] Row locks acquired
0709 13:12:55.426932 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:12:55.427053 (+   121us) write_op.cc:270] Start()
0709 13:12:55.427160 (+   107us) write_op.cc:276] Timestamp: P: 1783602775427013 usec, L: 0
0709 13:12:55.427175 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:12:55.434727 (+  7552us) log.cc:844] Serialized 532 byte log entry
0709 13:12:55.947112 (+512385us) op_driver.cc:464] REPLICATION: finished
0709 13:12:56.059498 (+112386us) write_op.cc:301] APPLY: starting
0709 13:12:56.060470 (+   972us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:56.094581 (+ 34111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:56.122165 (+ 27584us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:56.126364 (+  4199us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:56.126644 (+   280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:12:56.126719 (+    75us) write_op.cc:312] APPLY: finished
0709 13:12:56.173871 (+ 47152us) log.cc:844] Serialized 22 byte log entry
0709 13:12:56.226865 (+ 52994us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:56.290854 (+ 63989us) write_op.cc:454] Released schema lock
0709 13:12:56.290904 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:56.291299 (+   395us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:56.291337 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":123942,"reported_tablets":1,"spinlock_wait_cycles":29856000,"child_traces":[["op",{"apply.queue_time_us":110886,"apply.run_cpu_time_us":2988,"apply.run_wall_time_us":231802,"num_ops":1,"prepare.queue_time_us":469155,"prepare.run_cpu_time_us":2558,"prepare.run_wall_time_us":39611,"replication_time_us":519590,"thread_start_us":92714,"threads_started":1}]]}
I20260709 13:12:56.461966  2085 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.450498  2054 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:56.467497   923 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:12:56.516381   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58993.2us
I20260709 13:12:56.496380   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 225) took 1574 ms. Trace:
W20260709 13:12:56.485677   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.556764   749 rpcz_store.cc:276] 0709 13:12:54.921442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.921612 (+   170us) service_pool.cc:224] Handling call
0709 13:12:54.941995 (+ 20383us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02b47f3a 00007f0d088fe1cc 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc 00007f0d06ebc392 00007f0d00e7a48c 00007f0d00e798c6
0709 13:12:56.496158 (+1554163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11043,"reported_tablets":5,"spinlock_wait_cycles":22900480}
I20260709 13:12:56.468382  2142 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.467633   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:56.474757  2195 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.493513   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:56.525604  2095 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.571177   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.569355   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.550632  2046 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:12:56.575110   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:12:56.579897  2195 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.582231  2085 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:56.587050   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.589243  2195 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.593778   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.600273   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:56.600680  2129 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.601388  2129 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.602464   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.604118  2088 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:56.604207   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:56.606325  2160 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:56.606153  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:56.609414  2129 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:56.448478   750 rpcz_store.cc:276] 0709 13:12:54.450056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.450197 (+   141us) service_pool.cc:224] Handling call
0709 13:12:56.420977 (+1970780us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:54.906748 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:54.907593 (+   845us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:54.907670 (+    77us) write_op.cc:435] Acquired schema lock
0709 13:12:54.907701 (+    31us) tablet.cc:662] Decoding operations
0709 13:12:54.908100 (+   399us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:54.908170 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:54.908201 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:54.908388 (+   187us) tablet.cc:701] Row locks acquired
0709 13:12:54.908408 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:12:54.908538 (+   130us) write_op.cc:270] Start()
0709 13:12:54.908643 (+   105us) write_op.cc:276] Timestamp: P: 1783602774908503 usec, L: 0
0709 13:12:54.908665 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:12:54.933984 (+ 25319us) log.cc:844] Serialized 532 byte log entry
0709 13:12:55.200345 (+266361us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:12:55.424547 (+224202us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54000db4a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02ab66b8 00007f0d02ab62fa 00007f0d02b3a965 00007f0d02b3ae09 00007f0d06575581 00007f0d06574d58 00007f0d065799f1 00007f0d065799a9 00007f0d06579939 00007f0d06579901
0709 13:12:55.425368 (+   821us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3ae28 00007f0d06575581 00007f0d06574d58 00007f0d065799f1 00007f0d065799a9 00007f0d06579939 00007f0d06579901 00007f0d06578b2d 000000000038f354 000000000038f199
0709 13:12:55.861048 (+435680us) op_driver.cc:464] REPLICATION: finished
0709 13:12:56.061817 (+200769us) write_op.cc:301] APPLY: starting
0709 13:12:56.062019 (+   202us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:56.062181 (+   162us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:56.062216 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:56.062653 (+   437us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:56.110000 (+ 47347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:12:56.110094 (+    94us) write_op.cc:312] APPLY: finished
0709 13:12:56.110560 (+   466us) log.cc:844] Serialized 22 byte log entry
0709 13:12:56.110866 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:56.111043 (+   177us) write_op.cc:454] Released schema lock
0709 13:12:56.111114 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:56.115882 (+  4768us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:56.115938 (+    56us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":176045,"apply.run_cpu_time_us":3161,"apply.run_wall_time_us":69693,"num_ops":1,"prepare.queue_time_us":451398,"prepare.run_cpu_time_us":74930,"prepare.run_wall_time_us":518841,"raft.queue_time_us":798627,"raft.run_cpu_time_us":59040,"raft.run_wall_time_us":261475,"replication_time_us":842262,"spinlock_wait_cycles":140305280,"thread_start_us":815638,"threads_started":4,"wal-append.queue_time_us":159625}]]}
I20260709 13:12:57.017390   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:56.975751  2120 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:56.616703  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.018411  2120 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.019946   926 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:56.933849  2150 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:57.022415  2150 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.726569  2114 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.616813   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.023787   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:56.617450   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:56.595129  2095 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:56.648432   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.036857  2120 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:56.653707  2052 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:56.778347  2164 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.038489   929 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:56.929342  2046 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:57.003085  2115 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.039042   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.039495  2046 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.039636  2115 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.041769  2046 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.042603  2115 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.038507  2052 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.042869  2095 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:57.004236  1593 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.592s	user 0.014s	sys 0.001s
I20260709 13:12:56.622464  2142 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.919977   923 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:57.044538  2052 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.039710   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.038973  2164 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:56.872814  2088 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:57.005487  2138 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:56.836867   749 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.873224  2139 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.088572  2138 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.056725  2088 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.095645  2115 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.134970  2142 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.136325  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.138783  2102 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.139178  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.132151   998 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:57.096185  2139 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.092196  2114 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:12:57.153883   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 74736.6us
I20260709 13:12:57.140316  2046 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.139441  2102 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.132418  2115 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.155201  2150 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.174953  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.182327   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.192564  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.196763  1593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 271) took 1004 ms. Trace:
I20260709 13:12:57.167207   922 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:12:57.171160  2041 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.198777   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.178164  2150 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.173081  2046 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.208253  2088 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:12:57.210978   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 119826us
I20260709 13:12:57.167505   928 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.210726  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:57.214752  2041 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.215386  2102 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.217048  2041 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.216953   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.217625  2041 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.218182  2041 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.228309  2138 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.203903  2164 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.202286  2142 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.232786  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.219559  2041 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.243829  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.247151  2115 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.228487  2150 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.249311  2139 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.250505   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:57.250687  2142 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.252094  2200 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:57.252600  2200 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.253005  2200 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:57.253225  2142 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.255661  2046 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.258018  2164 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.261303  2164 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.263980  2199 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.264739  2199 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.266808  2199 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.267963   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.233672  2160 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.273921  2160 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:57.274355  2160 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.291834  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.294893  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.250514  2150 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.200728  1593 rpcz_store.cc:276] 0709 13:12:56.192154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.227641 (+ 35487us) service_pool.cc:224] Handling call
0709 13:12:56.292408 (+ 64767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.292816 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:57.043923 (+751107us) log.cc:844] Serialized 14 byte log entry
0709 13:12:57.196236 (+152313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.196299 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:57.196393 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.196509 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.196682 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":151503,"threads_started":1,"wal-append.queue_time_us":151926}
I20260709 13:12:57.290174   895 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:12:57.342942  2085 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.343426  2085 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:57.343817  2085 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.346530  2160 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.350487  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.308555  2164 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.356825   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.354650  1593 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.369771   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.370373  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.370427   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.371786   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.373292  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:12:57.374399   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 87776.9us
W20260709 13:12:57.375204   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 180266us
I20260709 13:12:57.377570  2114 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:57.378157   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.380841   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.379562  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.377575   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:57.387202   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.374723  2046 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.389281   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.389941  2046 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.391867  2041 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.392086  2046 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.387652  2202 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.387935   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.393949   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.388144  2142 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.374723  2160 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.375191  2164 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.398638  2164 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.400534   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.401041  2164 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.401620  2138 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:57.402215  2212 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.404291   821 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.405122  2150 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.406104  2150 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.406427  2150 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:57.406307  2200 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.407276  2150 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.408371   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.408504   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.408675  2150 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.409811   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.411995   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.422996   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 221) took 2276 ms. Trace:
I20260709 13:12:57.428850  2118 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.422257  2129 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.431712  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.426680   753 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.414656  2085 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:57.434592  2085 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.420126  2114 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.439158   753 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.442945   818 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:57.445369   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.445497  2102 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:57.426282  2142 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.450074  2214 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.450474  2214 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:57.450865  2214 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.451107  2142 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.453285   753 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.454906   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.455273   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.455920   847 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:57.456753  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.458201   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.462769  2200 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.463734   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.465080   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.466225   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.467439   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.458781   851 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:57.468571   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.469218   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:57.469765   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 73980.7us
I20260709 13:12:57.469908  2213 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:57.367928   848 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.474119   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.474145   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.474921  2213 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.475589  2164 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.477134   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.477967  2164 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.478821  2213 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.480582   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.480767   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.482962   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.434723  2215 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.485205  2046 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.485087  2164 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.490715   922 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:12:57.480647  2213 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.492910  2213 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.494029   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.494637   850 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:12:57.497493   928 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.493677  2120 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.499848   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.502192  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:57.504278   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.514982  2209 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.515388   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.515623   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.515010   923 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:12:57.516678   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.516930  2209 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.505448   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:12:57.507699   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 227590us
I20260709 13:12:57.517455  2118 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.517701   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.494889  2142 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:12:57.517967  2118 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.519356  2139 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.519719  2118 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.519798  2139 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.520279  2139 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.522500   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.524067   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.525233   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.521586  2203 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:12:57.528061  2120 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:12:57.528455   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.521751  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.532299  2115 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.532380  2211 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.533732   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.535782   969 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.538936  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.424943   751 rpcz_store.cc:276] 0709 13:12:55.138406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.138589 (+   183us) service_pool.cc:224] Handling call
0709 13:12:56.075851 (+937262us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02b47f3a 00007f0d088fe1cc 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc 00007f0d06ebc392 00007f0d00e7a48c 00007f0d00e798c6
0709 13:12:57.414766 (+1338915us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:57.132569 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:57.133255 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:57.144592 (+ 11337us) write_op.cc:435] Acquired schema lock
0709 13:12:57.146109 (+  1517us) tablet.cc:662] Decoding operations
0709 13:12:57.178473 (+ 32364us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:57.197244 (+ 18771us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:57.197299 (+    55us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:57.211894 (+ 14595us) tablet.cc:701] Row locks acquired
0709 13:12:57.212851 (+   957us) write_op.cc:260] PREPARE: finished
0709 13:12:57.213041 (+   190us) write_op.cc:270] Start()
0709 13:12:57.214699 (+  1658us) write_op.cc:276] Timestamp: P: 1783602777212971 usec, L: 0
0709 13:12:57.214740 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:12:57.215476 (+   736us) log.cc:844] Serialized 537 byte log entry
0709 13:12:57.288385 (+ 72909us) op_driver.cc:464] REPLICATION: finished
0709 13:12:57.380123 (+ 91738us) write_op.cc:301] APPLY: starting
0709 13:12:57.385186 (+  5063us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:57.392154 (+  6968us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:57.392272 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:57.392493 (+   221us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:57.392653 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:12:57.392701 (+    48us) write_op.cc:312] APPLY: finished
0709 13:12:57.392893 (+   192us) log.cc:844] Serialized 22 byte log entry
0709 13:12:57.393112 (+   219us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:57.393189 (+    77us) write_op.cc:454] Released schema lock
0709 13:12:57.393230 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:57.393579 (+   349us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:57.393613 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":691964160,"child_traces":[["op",{"apply.queue_time_us":76872,"apply.run_cpu_time_us":1678,"apply.run_wall_time_us":18091,"num_ops":1,"prepare.queue_time_us":545781,"prepare.run_cpu_time_us":3687,"prepare.run_wall_time_us":83229,"replication_time_us":73477,"thread_start_us":64122,"threads_started":2}]]}
I20260709 13:12:57.542183  2209 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:57.543267  2209 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.543710  2209 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.537647  2211 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.543282  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:12:57.552389  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.576687  1004 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:57.579596  2214 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.581127   850 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.589896   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:57.591259   925 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:12:57.596987   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.597193  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:57.600967  2214 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:12:57.602034  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 69604.2us
I20260709 13:12:57.602803  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.605101  2195 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.584813  2139 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.606889  2212 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:57.608902  2139 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.610669  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:57.611409  1002 raft_consensus.cc:606] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:57.613340  2209 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.615139  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:57.616079   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:57.616310   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:57.616694   848 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.616945   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:57.617520   850 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.617592  2150 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:57.618144  2150 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:12:57.619462  2118 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.621243  2150 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.622159   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:57.622011   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.622758   851 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.623865  1000 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.624706   848 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:57.626195  1600 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.629885   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.631793   969 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.632822   969 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.633531  2139 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.634222  2139 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.634670  2139 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:57.641973   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:57.644138   850 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:57.645308   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:57.646607  2211 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.647075  2211 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:57.649272  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:57.651258  1567 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.654767   851 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:12:57.635550  2139 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.662328  2220 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.663100  2220 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:12:57.650030  1001 raft_consensus.cc:606] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
W20260709 13:12:57.664904   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.648133  2211 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.666235  2220 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:12:57.677418   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 77022.6us
I20260709 13:12:57.656734  2118 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.656348  2209 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.685292   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.689042   819 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.690788   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.694754   923 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:12:57.697003   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:12:57.700809   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 214444us
I20260709 13:12:57.701251  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:12:57.700009  2214 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.719110   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.701617  2212 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.706219  2219 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.724960  2200 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.734115  2150 raft_consensus.cc:993] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:12:57.734593  2150 raft_consensus.cc:1081] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:12:57.737697   923 raft_consensus.cc:606] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:57.727520  2212 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:57.741539  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:57.742087  1001 raft_consensus.cc:606] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:57.750862   851 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:12:57.742036  2212 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.775054   851 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.779145   851 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:57.788087   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 69036us
I20260709 13:12:57.791118   896 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.794440  2150 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:12:57.793859   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70604us
I20260709 13:12:57.796406   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.802659   851 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.806659   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.807832   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.809418  2041 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.810618   925 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:12:57.812145   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.812920  2052 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.801213  2194 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:57.815331   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:12:57.816182  2052 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.814363  2194 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.818949  2202 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.821317  2202 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.826871  2202 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:57.833735  2041 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.796993  2150 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.840843  2150 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:57.841976  2229 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.843271  2221 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:57.848636   846 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.848771  2233 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.850663  1593 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:57.850998   850 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.851181   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.853152  2115 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:57.853523  2115 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:57.854296  2115 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.855916  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.856526   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:57.878791  1004 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:57.879849   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.881173   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:57.883730  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.855460  2115 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:57.882807   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:12:57.887229   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 62353.5us
I20260709 13:12:57.888867  2088 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:57.889031   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.827785  2202 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:57.891117   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:12:57.891894  1000 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.906322   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:57.907100   925 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.907662   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.908695   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:57.889050   852 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.896098   851 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.909363   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:12:57.897279  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.898036   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.893723   922 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:57.901213  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.901883   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.901976   750 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.903317  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.851850  2194 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:57.895996  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:57.888806  2201 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:12:57.904181  2165 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.904081  2220 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.904273   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:57.905530  2230 raft_consensus.cc:993] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:57.931836   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.933141  2213 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:12:57.934546  2212 raft_consensus.cc:1081] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:57.929282  2165 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:57.933043  2052 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:12:57.951408   855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:12:57.927222  2085 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:12:57.954134  2165 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:57.967695   855 raft_consensus.cc:606] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:12:57.952186   925 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:12:57.935873  2114 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:12:57.957283  1593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:12:57.985040   895 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.999908  2200 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.003959  2233 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:58.004510  2233 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:58.004186  2085 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:58.005649  2233 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:58.010814  2233 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.947350  2229 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.019177   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:57.969650  2202 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.957230   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:12:58.027025   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:12:58.024289  2052 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.066651   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:12:58.069219   998 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:58.069656   998 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:12:58.021286  1593 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:58.074935   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.075866  2214 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.076225   821 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.077507   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:12:58.093362   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.096323   995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:12:58.097100   995 raft_consensus.cc:606] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:12:58.097424   995 raft_consensus.cc:614] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:58.086874   998 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:58.097905   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 142369us
I20260709 13:12:58.099787   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:12:58.096451   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.105757   998 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.074898  2246 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:58.107767   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80820.3us
I20260709 13:12:58.087954  1593 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.094276  2102 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.095615  2213 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.104377  1595 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:58.112385  2221 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:58.113008  2165 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.120754  2221 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:12:58.129688  2228 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.135943   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8c347b6586f486192f48c46974a2526 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:12:58.137050  2205 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:58.139031   998 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:12:58.143895  2212 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:58.149489  1600 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:58.148413  2221 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:58.172104   848 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.175797 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:58.177788 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:58.190776   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82566.3us
W20260709 13:12:58.191133   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 82726.5us
I20260709 13:12:58.135242  1593 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:58.191648   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.194455   821 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.208757  2211 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:58.211400  2236 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.212139  2236 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:58.243816   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:12:58.244918   928 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:58.204703  2249 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.212663  2236 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:12:58.292992   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.284678  2142 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:58.282853  2214 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:58.285148   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:12:58.298012   750 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:12:58.290517  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 48353us
W20260709 13:12:58.330157   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46192.6us
I20260709 13:12:58.315788   848 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.308887  2214 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:58.380692  1593 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:58.548157  1000 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:58.552773  2214 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:58.669790  2236 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:58.581719  2211 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:58.671792  2211 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:58.694540  2211 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:12:58.695640  2115 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
W20260709 13:12:58.664986  2165 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
W20260709 13:12:59.100132  2144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.003s	sys 0.007s
W20260709 13:12:59.100871  2144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.004s	sys 0.007s
W20260709 13:12:59.233938  1567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.011s	sys 0.014s
W20260709 13:12:59.234478  1567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.011s	sys 0.014s
I20260709 13:12:59.297395  2266 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.298121  2266 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.317984  2266 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.470484  2266 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.471205  2266 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.474146  2266 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:12:59.643359   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 1.30681e+06us
W20260709 13:12:59.653898   821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.370s	user 0.019s	sys 0.043s
W20260709 13:12:59.654351   821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.371s	user 0.019s	sys 0.043s
W20260709 13:12:59.654585  1566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.278s	user 0.001s	sys 0.025s
W20260709 13:12:59.654681   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 1.37131e+06us
W20260709 13:12:59.657040  1566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.002s	sys 0.026s
W20260709 13:12:59.657424  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 1.28451e+06us
I20260709 13:12:59.714140  2228 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:59.714890  2228 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.718248  2271 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:59.719233  2271 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.720142  2269 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.720741  2269 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.721495  2270 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:59.722268  2270 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.723297  2269 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.725569  2271 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.725932  2270 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.717569  2228 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.726279  2271 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.727478  2271 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.729648  2271 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.732623  2138 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.733346  2138 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.735486  2138 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.742105  2138 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.742868  2138 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.745455  2271 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:59.747908  2274 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.748505  2274 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.747068  2138 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.749549  2271 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.750201  2271 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.752761  2271 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.754264  2276 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.754488  2138 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.755009  2276 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.755111  2138 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.757751  2138 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:12:59.758368  2271 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.759500  2271 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.761729  2271 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.763769  2232 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:59.763972  2273 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.764566  2228 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:12:59.764392  2232 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.765015   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1184) took 1521 ms. Trace:
I20260709 13:12:59.764657  2273 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.765218  2228 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.765479   848 rpcz_store.cc:276] 0709 13:12:58.243909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.246575 (+  2666us) service_pool.cc:224] Handling call
0709 13:12:58.289157 (+ 42582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:58.337061 (+ 47904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:59.764926 (+1427865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:59.766598  2232 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.767124  2232 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.767244  2269 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.767423  2273 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.767751  2272 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.768000  2273 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.767858  2269 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.768415  2272 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.768563  2273 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.770082  2269 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.770651  2272 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.771256  2251 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.771363  2232 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.771878  2251 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.773710  2232 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.773898  2251 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.774399  2251 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.774850  2251 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.761790  2203 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.771415  2273 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.775985  2203 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.776428  2278 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.767124  2228 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.777129  2278 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.778204  2203 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.778282  2276 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.762261  2138 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.779748  2138 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.782358  2203 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.784976  2251 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.785554  2251 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.786023  2251 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.782982  2203 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.789103  2203 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.789585  2203 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:59.790030  2203 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.792529  2138 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.793002  2138 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.793403  2138 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.794955  2282 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.795192  2138 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.815635  2281 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.815949   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.816253  2281 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.811486  2129 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.818279  2129 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:12:59.808910  2228 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:12:59.814881  2203 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.835439   995 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:59.841120  2274 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.842159   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.842754   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.835011  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.845883  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.847704   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.848573   995 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:12:59.848675  1002 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:59.851090  2203 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.851747  2203 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.842134  2278 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.853952  2138 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.854167  2278 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.854497  2138 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.817698  2251 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.854741  2278 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.843948  2280 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.858793  2275 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.859843  2138 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.852921  2283 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:12:59.867053  2214 raft_consensus.cc:993] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:12:59.795523  2282 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.826088  2139 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.866567  2280 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.873661   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.874254   998 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:12:59.868199  2275 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.872175  2120 raft_consensus.cc:1081] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:12:59.873493   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:59.862025  2281 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:12:59.876876   893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.589s	user 0.008s	sys 0.013s
I20260709 13:12:59.852990  2228 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.866073  2251 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:12:59.820536  2129 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.861961  1597 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:12:59.875319  2279 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:12:59.891641   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.892272   996 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:12:59.893066   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.893620   996 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:12:59.894074  2278 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.886895  2280 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.887399  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.888404  2129 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:59.888624   893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.603s	user 0.009s	sys 0.014s
I20260709 13:12:59.894423   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.899408  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:12:59.869376  2283 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:12:59.884933   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:12:59.879987  2273 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:59.897464   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:12:59.873330  2139 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.227119  2277 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.268959   996 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:00.269237  2220 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:00.269894  2220 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.270426  2220 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.274811  2283 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:00.269300  2277 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.279121  2277 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:12:59.989679  2285 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.284610  2285 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.287715  2285 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:12:59.993672  2275 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.994912  2282 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:12:59.950560  2203 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:00.279728  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 380362us
I20260709 13:13:00.281553  1002 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.348038  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.349917  1002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:13:00.335405  2220 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:12:59.950752  2228 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:00.452431   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 554990us
I20260709 13:13:00.452600  2120 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:59.899273  2251 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.454772  2286 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.455376  2273 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.460278  2052 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.472862  2230 raft_consensus.cc:993] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:12:59.899689  2200 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.548009  2052 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.548395  2200 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.549494  2194 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.550235  2052 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.550014  2194 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.550668  2200 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.483883   998 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
W20260709 13:13:00.494637   894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.001s	sys 0.015s
W20260709 13:13:00.551391   894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.002s	sys 0.015s
I20260709 13:13:00.551842  2286 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:00.506698   896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.003s	sys 0.014s
I20260709 13:13:00.529127   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1155) took 2321 ms. Trace:
W20260709 13:13:00.533985   855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.210s	user 0.025s	sys 0.015s
W20260709 13:12:59.950860   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.557s (SENT) [suppressed 1 similar messages]
I20260709 13:13:00.503238  2294 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.479426  2139 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:00.540325  1597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.004s	sys 0.015s
I20260709 13:13:00.544808  2298 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.547696  2230 raft_consensus.cc:1081] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:13:00.548756  2220 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.533107  2295 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.537278  1002 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:13:00.554280   896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.003s	sys 0.014s
I20260709 13:13:00.597128  1002 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.598127  2289 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.601183  2289 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.603279  2289 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.605578  2290 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.605926   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.606223  2114 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.606132  2290 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.606806  2114 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.543573  2120 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.607955  2120 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.608342  2290 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.608904  2290 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.608979  2114 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:00.609359  2290 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.609889  2114 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.610543  2114 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.611545  2290 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.612798  2114 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:00.613992   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.546995  2273 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:00.568293  2286 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.607191  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 62) took 2505 ms. Trace:
I20260709 13:13:00.570259  2251 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:00.603734  2294 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.623085  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.625146   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:00.598541  1597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.004s	sys 0.016s
I20260709 13:13:00.631276  2290 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.640455  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 300) took 2631 ms. Trace:
I20260709 13:13:00.609719  1594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 299) took 2601 ms. Trace:
I20260709 13:13:00.640738  2290 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.577469  2052 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.631619  1003 rpcz_store.cc:276] 0709 13:12:58.101320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.101843 (+   523us) service_pool.cc:224] Handling call
0709 13:13:00.606579 (+2504736us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b5400088328. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07ae3ebc 00007f0d07b1908c 00007f0d07b0393b 00007f0d07b033d2 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c
0709 13:13:00.606701 (+   122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.606864 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.606902 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:00.606944 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:00.606973 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.607027 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.607138 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4773760}
I20260709 13:13:00.637127   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.828140  2052 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.637192   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.638144  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.638415  2294 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:00.644088   893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.014s	sys 0.010s
W20260709 13:13:00.830338   893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.014s	sys 0.010s
W20260709 13:13:00.645074   855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.321s	user 0.027s	sys 0.022s
I20260709 13:13:00.831483   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1185) took 2526 ms. Trace:
I20260709 13:13:00.831856   855 rpcz_store.cc:276] 0709 13:12:58.305477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.305609 (+   132us) service_pool.cc:224] Handling call
0709 13:12:58.315123 (+  9514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:58.319274 (+  4151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:58.319320 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:58.320102 (+   782us) log.cc:844] Serialized 26 byte log entry
0709 13:13:00.831030 (+2510928us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:00.831069 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.831132 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.831169 (+    37us) raft_consensus.cc:1685] finished
0709 13:13:00.831180 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.831291 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7496,"raft.queue_time_us":3858,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":9444,"thread_start_us":2506607,"threads_started":1,"wal-append.queue_time_us":250539,"wal-append.run_cpu_time_us":2241,"wal-append.run_wall_time_us":1008003}
I20260709 13:13:00.833680   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.834220   855 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:00.597374   999 rpcz_store.cc:276] 0709 13:12:58.207569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.207746 (+   177us) service_pool.cc:224] Handling call
0709 13:12:58.215760 (+  8014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:58.242331 (+ 26571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:58.242356 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:58.246891 (+  4535us) log.cc:844] Serialized 494 byte log entry
0709 13:12:58.388815 (+141924us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:58.388858 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.388945 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.528846 (+2139901us) raft_consensus.cc:1685] finished
0709 13:13:00.528868 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.529034 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":100385,"threads_started":1,"wal-append.queue_time_us":330209,"wal-append.run_cpu_time_us":1831,"wal-append.run_wall_time_us":1001796}
I20260709 13:13:00.628680  2295 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.836170  1599 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.837235  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:00.837975  2295 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.646863  1002 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.635257  2286 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.628454  2298 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:00.664541   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 713629us
I20260709 13:13:00.658562  2288 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.670339   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:00.671267   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47765.8us
I20260709 13:13:00.677558  2290 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.642490  1594 rpcz_store.cc:276] 0709 13:12:58.008824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.009472 (+   648us) service_pool.cc:224] Handling call
0709 13:13:00.608940 (+2599468us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:00.609151 (+   211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.609329 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.609359 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:00.609408 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:00.609442 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.609514 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.609637 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15255296}
I20260709 13:13:00.673406  2292 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.672025  2266 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.691592  2304 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.640888  1597 rpcz_store.cc:276] 0709 13:12:58.009225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.009892 (+   667us) service_pool.cc:224] Handling call
0709 13:12:59.861360 (+1851468us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:12:59.861604 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:59.861834 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:59.971153 (+109319us) log.cc:844] Serialized 14 byte log entry
0709 13:13:00.640086 (+668933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:00.640148 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:00.640191 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.640271 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.640384 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10540,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":2478,"spinlock_wait_cycles":14515072,"thread_start_us":668529,"threads_started":1,"wal-append.queue_time_us":569592}
I20260709 13:13:00.694895  2246 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.698261  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:00.830375  2052 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.701906   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.696609  2273 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.704346   851 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.710322  2299 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.715883  2303 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.715790   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.720769   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.900794   846 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:00.723045   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.715852   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.732323   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.738128  2305 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.737512  2302 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.741868  2307 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.737671  2306 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.743494   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 114) took 2766 ms. Trace:
I20260709 13:13:00.746853   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.752890  2129 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.754863  2220 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.771183  2200 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.802990  2214 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.803752  2310 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.808109   925 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.809267   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:00.907828  1000 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:00.814882   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.816130   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.816497   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.817219   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.818046   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.818650   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.818813   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.837944  1596 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:00.924794  2303 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:13:00.650484  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 342310us
I20260709 13:13:00.842958  1002 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:00.939296  2305 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.702013  2041 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:00.901409   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54725.6us
I20260709 13:13:00.901731   854 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:00.867208  2290 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:00.848253  2286 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.940524  2213 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.945246   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:00.946698  2286 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:00.945808  2213 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.813633  2233 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.882135  2266 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.947515  2233 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.941437   926 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.899726  2279 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.949011  2266 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.950713  2298 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:00.928267   849 raft_consensus.cc:2368] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:00.936640  2214 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.930047  2310 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.924111  2129 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.953141  2214 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.953267  2129 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.953441   853 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:00.929961  2220 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.943245  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:00.954164  2129 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:00.954564  2129 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.954816  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:00.955449  2220 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:00.956037  2220 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:00.956521  2295 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:00.956698  2194 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.956890  2295 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.957331  2295 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.924376  2200 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.849134   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.928828  2302 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.930445  2236 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:00.852028  2288 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.946883   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:00.943519  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51128 (request call id 1204) took 1058 ms. Trace:
I20260709 13:13:00.944690   923 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:00.962262   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:00.944152   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:00.930279  1000 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:00.963930  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:00.943351   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:00.943939  2041 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.966307  2233 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:00.945106  2211 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:00.882709  2292 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:00.949536  2279 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.912942  2246 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.911677  2304 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.930500  2306 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:00.945003  2114 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.209s
I20260709 13:13:01.155331  2114 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:01.155875  2114 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:01.156564  1000 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:01.000185   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 157209us
I20260709 13:13:01.123013   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.085671   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.165473   999 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.024508  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.158625  1004 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.076006   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:01.140105   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 270771us
I20260709 13:13:01.022495  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.168305   850 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.167080  1601 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:01.156893  2114 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.041841   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.969349  1002 rpcz_store.cc:276] 0709 13:12:59.888100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.349790 (+461690us) service_pool.cc:224] Handling call
0709 13:13:00.943427 (+593637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:00.921955  2299 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.052615   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.194962   854 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:00.914846   750 rpcz_store.cc:276] 0709 13:12:57.977297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.983382 (+  6085us) service_pool.cc:224] Handling call
0709 13:13:00.743412 (+2760030us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:58.338514 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:58.345589 (+  7075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:58.345623 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:12:58.345638 (+    15us) tablet.cc:662] Decoding operations
0709 13:12:58.345957 (+   319us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:58.346007 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:58.346039 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:58.346211 (+   172us) tablet.cc:701] Row locks acquired
0709 13:12:58.346227 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:12:58.346347 (+   120us) write_op.cc:270] Start()
0709 13:12:58.346463 (+   116us) write_op.cc:276] Timestamp: P: 1783602778346309 usec, L: 0
0709 13:12:58.346484 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:12:58.347052 (+   568us) log.cc:844] Serialized 595 byte log entry
0709 13:12:59.101176 (+754124us) op_driver.cc:464] REPLICATION: finished
0709 13:12:59.102382 (+  1206us) write_op.cc:301] APPLY: starting
0709 13:12:59.102452 (+    70us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:59.102574 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:59.102693 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:59.102940 (+   247us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:59.103089 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:12:59.103119 (+    30us) write_op.cc:312] APPLY: finished
0709 13:12:59.103312 (+   193us) log.cc:844] Serialized 23 byte log entry
0709 13:12:59.103474 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:59.103557 (+    83us) write_op.cc:454] Released schema lock
0709 13:12:59.103600 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:59.103943 (+   343us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:59.103973 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38038,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":982,"apply.run_cpu_time_us":1524,"apply.run_wall_time_us":1524,"num_ops":1,"prepare.queue_time_us":154,"prepare.run_cpu_time_us":2434,"prepare.run_wall_time_us":36117,"replication_time_us":754511,"thread_start_us":593,"threads_started":1}]]}
I20260709 13:13:00.927508  2311 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:00.954919  2129 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.942986  2313 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:00.958606  2220 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:00.968343  2296 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:01.131480  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:00.953156  2310 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.206135  2305 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.206591  2233 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.206940  2041 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:01.207791   926 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:01.079552  2289 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.209138  2213 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:01.190575  1000 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.210892   925 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:01.212802  1592 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:01.169366   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.271983   847 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.273142   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.274341   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.078958  2314 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.275405  2314 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.277109  2114 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.972467  2228 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.001912  2286 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.066087  2201 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.028131  2236 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.247193   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.207369  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:01.220927   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52100.7us
I20260709 13:13:01.215277  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.254622   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.240244   749 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 1 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.258788   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:00.928524  2307 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.106359  2120 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.256402  2303 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:01.277197   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.244511   753 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.305914   753 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.284848  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.308522  1002 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.214176  2220 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.278867  2289 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.263685  2233 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.287149  1595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:13:01.214809  2273 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.220191  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:01.229103   895 connection.cc:256] RPC call timeout handler was delayed by -1.7140621240000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 13:13:00.945039  2290 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.213476  2313 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.241490  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.297744   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.257349  2310 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.311013   753 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.277642  2314 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.288640  2315 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.335273  2315 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:01.326094   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 134925us
I20260709 13:13:01.287135  2201 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.287539  2286 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.320106  2327 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.326037  1595 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:13:01.329197  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46475.3us
I20260709 13:13:01.214349  2311 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.361625 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:01.293326  2228 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.345572   846 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.370568  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:01.342310   848 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:01.381278  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:01.327893  2303 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:01.395582   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 104289us
W20260709 13:13:01.413630   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.340500  2052 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:01.342550  2316 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:01.251792  2296 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.366514  2200 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.376091  1595 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.428061  2294 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:01.457207   968 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:01.425458  2318 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.341426  2310 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.463006  2322 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.467952  2302 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.445698  2273 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:01.437608  2274 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:01.426309  2246 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.375124  2327 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.487687  2294 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:01.535840  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 589485us
I20260709 13:13:01.519068  2296 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:01.446599  2288 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.470043  2325 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:01.524456   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 240090us
I20260709 13:13:01.442008  2052 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:01.543051   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 167966us
I20260709 13:13:01.545537  2299 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:01.554999   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 109046us
W20260709 13:13:01.532989  2295 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.575s	user 0.020s	sys 0.029s
I20260709 13:13:01.560061  1004 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:01.561902   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.554580  2325 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.562772   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.449568  2316 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.531172  2304 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.524540  2318 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.427037  2303 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.534291  2256 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:01.563354   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.870930  2228 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.870893  2256 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:01.871830   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:01.725389   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 349405us
I20260709 13:13:01.542140  2294 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:01.517627  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57085us
I20260709 13:13:01.874046   855 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:01.588342   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.534317  2120 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.587430   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:01.678929   849 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1174) took 3683 ms (client timeout 1667 ms). Trace:
W20260709 13:13:01.698829   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.674434  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.879410   999 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:01.530205  2274 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:01.847656   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 53) took 1032 ms. Trace:
I20260709 13:13:01.845399  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:01.881134   928 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:01.886242   855 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:13:01.883158  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:01.926115  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 80837.7us
I20260709 13:13:01.883591   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:01.604200   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49019.8us
I20260709 13:13:01.898447  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:01.924167   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51378.4us
I20260709 13:13:01.677179   751 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:01.927758  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:01.880272   929 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:01.691778   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.887537   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.926394   930 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:01.929404   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.930737   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.928608   751 catalog_manager.cc:5697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:01.934406  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.933763   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:01.936187   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 415837us
I20260709 13:13:01.936343  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.937749   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.925606   923 rpcz_store.cc:276] 0709 13:13:00.815759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.815880 (+   121us) service_pool.cc:224] Handling call
0709 13:13:01.847387 (+1031507us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b540028ffd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:01.847560 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25134848}
I20260709 13:13:01.871951  2256 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.938979   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.939810   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:01.944288   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.944393   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:01.944550   923 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:01.587440  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.539660  2273 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.933811   924 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:01.872094  2290 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:01.636726  2292 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.873871  2315 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.590924  2311 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:01.723191  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.934172  2298 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:01.944612   998 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:01.947533   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.944978   996 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.947741  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.674861  2307 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:01.940244   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 65159.2us
I20260709 13:13:01.934662  2236 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:01.938757  2194 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:01.951855   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.722894  2306 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:01.928162   849 rpcz_store.cc:269] 0709 13:12:57.880408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.886775 (+  6367us) service_pool.cc:224] Handling call
0709 13:13:01.563921 (+3677146us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b540019e6d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:01.564061 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6929842048}
W20260709 13:13:01.667385  1622 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1060 after lost signal to thread 32227
I20260709 13:13:01.931385  2274 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:01.955960  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.943940   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.956664   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.960492  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:01.952875  1594 raft_consensus.cc:1813] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Handling vote request from an unknown peer 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:01.962770   930 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:01.884363  2295 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:01.953876  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.945423   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.674715   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.546721  2322 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:01.975328   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:01.986061   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:01.978415   997 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:01.982626   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:01.978044  1001 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:01.993939   929 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:02.021832   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:02.025846   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 50624.7us
I20260709 13:13:02.027436   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.030532   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:01.976038  2289 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:02.019901  2129 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.011s
W20260709 13:13:02.031356   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67150.1us
I20260709 13:13:01.989140  2201 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.032933   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:02.039001   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:02.033468   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:01.989707  2286 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.031589  1598 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:13:02.026299  1000 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.756s	user 0.006s	sys 0.014s
I20260709 13:13:02.023756  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:01.986649  2120 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:13:02.035825  1622 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1740 after lost signal to thread 787
I20260709 13:13:02.035555  1594 raft_consensus.cc:2410] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:02.037792  2312 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:01.982782  2341 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:13:02.049882   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.003877  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.051887  2319 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.066622  2303 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.031352  2334 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.067938  2052 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.069259  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.081127   998 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
W20260709 13:13:02.071126  2211 consensus_meta.cc:287] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.916s	user 0.009s	sys 0.001s
I20260709 13:13:02.093393   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:02.071962  2220 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.012s
I20260709 13:13:02.073179  2295 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.076576   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.025830  2323 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.045246  1000 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:02.041559  2129 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.085355   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.083784  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.043985  2289 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.082770  2277 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:13:02.105685   929 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:02.106336  1000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33108 (request call id 64) took 2375 ms (client timeout 2142 ms). Trace:
W20260709 13:13:02.106894  2114 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.829s	user 0.043s	sys 0.043s
I20260709 13:13:02.101686  2052 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.107625  2052 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.107992  2052 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.083736  2319 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:02.096422   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:02.109664   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.091955  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.095544   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:02.110525   996 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:02.111224   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:02.083874  2299 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:02.080761  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:02.095988   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57054.3us
I20260709 13:13:02.100854  2342 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:02.106720  1000 rpcz_store.cc:269] 0709 13:12:59.730805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.697877 (+967072us) service_pool.cc:224] Handling call
0709 13:13:02.106253 (+1408376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:02.099948  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:02.118110   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 143653us
I20260709 13:13:02.115482   819 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:02.120873   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.121528   930 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:02.121774  2114 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:02.122397   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.099663  2120 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.118851  1002 raft_consensus.cc:2376] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2. Re-sending same reply.
I20260709 13:13:02.127211   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.092978  2312 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.129112   929 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.131656   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.116299  2306 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.119405  2319 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.139007   921 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:02.139379  2317 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:02.129413  1622 debug-util.cc:398] Leaking SignalData structure 0x7b08001df620 after lost signal to thread 879
I20260709 13:13:02.148929  1598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 291) took 1007 ms. Trace:
I20260709 13:13:02.128702  2120 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:02.149317  1598 rpcz_store.cc:276] 0709 13:13:01.141572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.169391 (+ 27819us) service_pool.cc:224] Handling call
0709 13:13:02.148730 (+979339us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b5400591d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:02.148871 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668744960}
I20260709 13:13:02.149787  1597 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.150604  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.151111  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:02.151671  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.152050   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:02.152310  1597 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.124095  2299 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.153165   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.139930  2129 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.123835  2295 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.141716  2331 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.164764   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.101153  2323 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.136381  2310 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.125779  2342 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:02.144343   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:02.235636   930 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:02.119048  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 136405us
I20260709 13:13:02.098388  2220 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.147181   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.239039   929 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.131497  2114 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.240868  2114 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.241024   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.168401  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.223091  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.168599   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:02.227526   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 66194.6us
I20260709 13:13:02.136596  2312 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.232462   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.236330  2260 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:13:02.149806  2120 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.164045  2295 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.232437  2323 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:02.149351  1622 debug-util.cc:398] Leaking SignalData structure 0x7b080031c120 after lost signal to thread 954
I20260709 13:13:02.238085  2342 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.159797   895 leader_election.cc:400] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:02.246135   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.241310  2114 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.246763  2348 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.252223  1599 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.242992   927 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.257786  2260 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:02.258062  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.256685   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.259372   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.261839   895 leader_election.cc:403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:02.261896   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:02.262784   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 135965us
I20260709 13:13:02.242981   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.260149   929 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.268044   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.265962   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.251042  2295 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.247435  2305 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.270098   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.244843  2211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51112 (request call id 74) took 4597 ms. Trace:
I20260709 13:13:02.272282   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.273221  2349 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.271034   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.276857   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.275326   854 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.278892   846 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.279466   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.281760   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.276439   925 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:02.264708  2322 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.284214   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.286993   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.264978  2260 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:02.285589   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.286823   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.288224   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 48) took 1486 ms. Trace:
I20260709 13:13:02.290376  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.289644   929 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:02.289470   927 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:02.274823  2314 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.292346   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.274220  2348 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.296630  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.290522   922 rpcz_store.cc:276] 0709 13:13:00.802139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.840228 (+ 38089us) service_pool.cc:224] Handling call
0709 13:13:00.902865 (+ 62637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.364971 (+462106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.368446 (+  3475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:01.375668 (+  7222us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:01.383156 (+  7488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.283510 (+900354us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54000bc554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:02.283556 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:02.288142 (+  4586us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007364480}
I20260709 13:13:02.309926   929 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:02.319038  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79012.2us
I20260709 13:13:02.269934  2236 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:02.264957  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:02.333966  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83017.9us
I20260709 13:13:02.335254  1566 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.284858  2331 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:02.292758  1622 debug-util.cc:398] Leaking SignalData structure 0x7b0800337420 after lost signal to thread 1028
I20260709 13:13:02.331990   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.332486   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.292927  2115 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.291260  2260 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.294711  2314 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.361485   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.282138  2305 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.363130   854 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.367527  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.364185   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:02.368875   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.368954   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:02.371119   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.374581   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.376116   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.361872  2277 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.361997  2314 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.331476  2052 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.386291  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.386063   927 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:02.367830  2194 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.383883   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.386744   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.375221  2214 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:02.367918  2115 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.387557   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.378753  2306 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:02.405567   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.408669   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.408960   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.408669   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.410235  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.411049   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.411711   998 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:13:02.413148   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.402055  2194 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.416246   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.404261  2115 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.417455   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.418180   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:02.351794  2120 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.422096  2347 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:02.423918  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.424654  1002 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:02.417132  1622 debug-util.cc:398] Leaking SignalData structure 0x7b0800367060 after lost signal to thread 1625
I20260709 13:13:02.439090   998 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.440719   998 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.412297  2306 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:02.417420  2194 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.420760   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1185) took 1716 ms. Trace:
I20260709 13:13:02.378450  2331 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.422689  2347 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.457109  2120 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:02.459254   967 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.557s (SENT)
I20260709 13:13:02.459786   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.461253  2346 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.462054  2342 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.463464  1598 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.465240   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:02.485066   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.588457  2305 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.588341  2306 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.591065  2331 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:02.591434  2354 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:13:02.448146  2220 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:02.520323   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 247966us
I20260709 13:13:02.595800   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.595175   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.612780   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.599517   930 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:02.595253  1002 raft_consensus.cc:2955] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:02.606951  2345 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:02.606966  2339 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:02.608508  2348 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.618615  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.619691  2290 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:02.619467  2345 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.622320  2345 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.622991  2339 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.610404   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.442533  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.615867   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.624958   854 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.632238   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.633790   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.635926   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:02.636596  2256 consensus_meta.cc:287] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.002s
I20260709 13:13:02.637087  2256 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.637712   928 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:02.638315   995 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.639218   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:02.652284  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 231254us
I20260709 13:13:02.653575  2233 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.653863  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.654987  2298 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.655695  2298 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.656570  2316 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.657667  2279 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.606380  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:02.666718   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.667448   851 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.673035   929 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.633791  2339 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.666711  2279 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.679216  2260 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:02.666163   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50361.4us
I20260709 13:13:02.681209   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.451949   848 rpcz_store.cc:276] 0709 13:13:00.702918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.707945 (+  5027us) service_pool.cc:224] Handling call
0709 13:13:00.708332 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.395566 (+687234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.408827 (+1013261us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54000fe954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:02.419122 (+ 10295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432016384}
W20260709 13:13:02.695143   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 316220us
I20260709 13:13:02.683445   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.684002   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:02.684319  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.701503   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:02.685523  2114 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.671631  2273 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:02.691866  2319 consensus_meta.cc:287] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.000s
I20260709 13:13:02.277704  2211 rpcz_store.cc:276] 0709 13:12:57.641744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.665781 (+ 24037us) service_pool.cc:224] Handling call
0709 13:12:57.834354 (+168573us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b2c000e6738. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01447a0a 00007f0d01447693 00007f0d01447631 00007f0d02b7a902 00007f0d02b7a6dd 00007f0d02b45f3f 00007f0d02b456b3 00007f0d02b3d9fa 00007f0d07b054c9 00007f0d01081c54
0709 13:12:57.890616 (+ 56262us) spinlock_profiling.cc:243] Waited 9.07 ms on lock 0x7b5800291ed0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02a9b0aa 00007f0d02b1d447 00007f0d02b3a70a 00007f0d02b3b76a 00007f0d02b3a91b 00007f0d02b4603c 00007f0d02b456b3 00007f0d02b3d9fa 00007f0d07b054c9 00007f0d01081c54
0709 13:12:57.908793 (+ 18177us) log.cc:844] Serialized 494 byte log entry
0709 13:12:57.964824 (+ 56031us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b180014bcc0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:12:58.179493 (+214669us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b180014bcc0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa52522 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:12:58.346784 (+167291us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b180014bcc0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:12:59.077528 (+730744us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54000e7ca0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab4dbe 00007f0d02abf930 00007f0d02abf8d8 00007f0d02abf858 00007f0d02abf813 00007f0d02abea9c 00007f0d089aa626 00007f0d089a4dd8 00007f0d02afb7cb 00007f0d02afe2c0
0709 13:13:00.938275 (+1860747us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b18002d2040. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b7ae3b 00007f0d02b22938 00007f0d02b3c2d4 00007f0d02ac1f14 00007f0d02ac1ec3 00007f0d02ac1e38 00007f0d02ac1df3 00007f0d02ac10dc 00007f0d02aeae16 00007f0d02ad2cbb
0709 13:13:02.100430 (+1162155us) log.cc:844] Serialized 14 byte log entry
0709 13:13:02.235819 (+135389us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8274,"raft.queue_time_us":43927,"raft.run_cpu_time_us":2387,"raft.run_wall_time_us":80784,"spinlock_wait_cycles":1719942400,"thread_start_us":133845,"threads_started":2,"wal-append.queue_time_us":194241,"wal-append.run_cpu_time_us":24439,"wal-append.run_wall_time_us":2113711}
I20260709 13:13:02.700322   851 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:02.710883   851 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:02.701681  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 173) took 1161 ms. Trace:
I20260709 13:13:02.624289  2052 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.716694   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.692950  2279 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.703001   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.704761  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.707939   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.705308  2041 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.725926   928 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:02.722904   851 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.729527   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.731303   851 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.714529  2331 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.731304   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.732846   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.733199   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 90a3708334cc4bd59c73d494d2747d54 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:13:02.721431  2279 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.706902  2323 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:02.746014   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.729307  2041 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.729135   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.751430   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.751574   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.804596   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.731057  2302 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:02.734211  2331 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.718819  2319 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.750833   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.812610   929 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.813743   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.748090  2213 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:02.744388  2279 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.728222  2295 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:02.806002  2294 consensus_meta.cc:287] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.004s
I20260709 13:13:02.714071  1596 rpcz_store.cc:276] 0709 13:13:01.540702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.542075 (+  1373us) service_pool.cc:224] Handling call
0709 13:13:02.672840 (+1130765us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:02.701599 (+ 28759us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26243712}
I20260709 13:13:02.807029  2041 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.815291  2318 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:02.815761  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 156672us
I20260709 13:13:02.852406  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:02.811992  1595 consensus_meta.cc:287] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 1.339s	user 0.015s	sys 0.009s
I20260709 13:13:02.836652   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.803752  2052 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.860347   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.839881   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.808334  2313 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:02.806798  2312 consensus_meta.cc:287] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.002s
I20260709 13:13:02.830791  2331 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.808666  2214 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:02.842365  2294 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.844444  2347 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.845374   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.875406  1594 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.875593   847 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.876686   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.877239  2294 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.878394  2277 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.878206  2194 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.869302  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.832903  2213 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.835857   851 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:02.850988   846 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:02.880357  2213 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:02.869436  2115 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.881175   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.882830   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.882187  2115 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.883795   854 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:02.884217  2194 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:02.887442   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.877838   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.889101   847 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.899634  2194 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.899140  2314 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.900542  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:02.902068  2327 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:02.902153  2314 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.906795   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.907590   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:02.907910   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.908241   851 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.909010   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.814234  2302 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:02.858748   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:02.861444   852 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.863931   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.864790  2052 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.865980  2299 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.866221   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:02.866763  2350 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.152s	user 0.000s	sys 0.003s
I20260709 13:13:02.867198  2323 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.867815  2325 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.871132  2318 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:02.872763  2114 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:02.873750  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:02.880061  2274 consensus_meta.cc:287] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.902s	user 0.013s	sys 0.008s
I20260709 13:13:02.912683   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:02.995564   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.964216  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.996345   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:02.863514  1595 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:02.913427   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:02.927229   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:02.929462   996 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:02.998797   928 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:02.953045   851 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:02.919278  1564 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.920429   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.915409   855 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:02.955180   997 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:02.961666   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.925678  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.920347   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.899089  2295 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:03.002095  1600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 174) took 1453 ms (client timeout 1563 ms). Trace:
W20260709 13:13:03.000357  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81124.3us
I20260709 13:13:03.004552   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.004745  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:02.890702  2041 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:02.913812   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.006533   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:02.921362  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:02.869647  2312 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.003350  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 178) took 1277 ms. Trace:
I20260709 13:13:02.963819  2302 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:03.005750  1600 rpcz_store.cc:269] 0709 13:13:01.549243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.564152 (+ 14909us) service_pool.cc:224] Handling call
0709 13:13:02.142682 (+578530us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:02.925517 (+782835us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ae9c 00007f0cffa1aa5b 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:03.002029 (+ 76512us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64788096}
W20260709 13:13:02.913976  2335 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:13:03.018289   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.026068   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.007027   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.027004   930 raft_consensus.cc:2368] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4.
I20260709 13:13:02.917645   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.019519   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.023200   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.005414  2299 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:02.975734  2346 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:13:03.036420   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.024945  1599 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:03.028824   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.029412   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:02.916512  2339 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:13:03.042629   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.043849   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.029373   846 raft_consensus.cc:2376] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2. Re-sending same reply.
I20260709 13:13:03.045238  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.003067  2296 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:03.046059  1593 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.046135  2296 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.047510   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.015714  2298 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.032246  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.022787  1601 rpcz_store.cc:276] 0709 13:13:01.725769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.961479 (+235710us) service_pool.cc:224] Handling call
0709 13:13:02.873623 (+912144us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ae9c 00007f0cffa1aa5b 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:03.003296 (+129673us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4788736}
I20260709 13:13:03.053397   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:03.029995   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 69519.9us
I20260709 13:13:03.045301   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.050280   923 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:03.043949   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.054800   852 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.055327   927 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:03.056527  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.056473  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.003420  2274 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.029194  2362 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.065862  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.040577  2294 raft_consensus.cc:2764] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:02.920043  2214 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.072326   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.077139   896 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:03.045285  1592 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.037025   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.058975   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:03.070241   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.078913   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.079577   924 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:03.079800 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:03.058149  2298 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:03.080811   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.071853  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.024592  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:03.085057  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 68392.8us
I20260709 13:13:03.086043  1597 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.069979   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.072109  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.071097   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.081354   924 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:03.092612   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.092835   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.093633   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.093895   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.094235   924 raft_consensus.cc:2376] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4. Re-sending same reply.
I20260709 13:13:03.094694   923 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:03.095028   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.095410   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.096065   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.093660   847 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.096776  2323 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.097209  2323 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:03.097250   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.098428   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.099195  1598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 314) took 1020 ms. Trace:
I20260709 13:13:03.084923  2298 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.099581  1598 rpcz_store.cc:276] 0709 13:13:02.079077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.151519 (+ 72442us) service_pool.cc:224] Handling call
0709 13:13:02.237492 (+ 85973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:02.513929 (+276437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.098953 (+585024us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400700154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:03.099137 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513719680}
I20260709 13:13:03.099535   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.101719   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.082511  2327 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.102702   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:03.104171   967 leader_election.cc:336] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.500s (SENT)
I20260709 13:13:03.104640   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.078605  2362 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:03.105392  2346 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:03.105753   967 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.514s (SENT)
I20260709 13:13:03.105703  2362 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.106843   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.107587  2346 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.108282  2342 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.104512  2041 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.078967  2345 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.108805   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.109220  2345 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.111114   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:03.111238  2345 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.085776  2277 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.076148  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.061256   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.141077  2277 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:03.141201  1593 raft_consensus.cc:1813] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Handling vote request from an unknown peer 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.142056  1593 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:13:03.077934  2350 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:13:03.143818   926 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.096127   925 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:03.096364   930 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:03.145572   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1224) took 1097 ms. Trace:
I20260709 13:13:03.145520   894 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:03.146265   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.146536  2289 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.146799   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.086961   848 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:03.111752   851 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.111903  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.174436  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.113238  2288 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:03.114336  2318 consensus_queue.cc:1059] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:03.071558  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.082852   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.116627  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.181874  1004 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.085772  2312 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.095468   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.184070   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.185402   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1227) took 1243 ms. Trace:
I20260709 13:13:03.081629   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.126652  1595 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.126564   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.115856  2334 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:13:03.127187  2327 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:03.115859  2346 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.128506  1596 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:03.129158   921 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 60) took 2309 ms (client timeout 1560 ms). Trace:
W20260709 13:13:03.130184  2364 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:13:03.131009  2213 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.131839   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:03.133064   967 leader_election.cc:336] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.785s (SENT)
I20260709 13:13:03.217852   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:03.218937   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85898us
I20260709 13:13:03.220175   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.135721   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.137805  2052 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.139101  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.140319   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:03.224124   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 83884.1us
I20260709 13:13:03.225908   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:03.088454   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.141264   852 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.227783  2357 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.228206  2357 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:03.228564  2357 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.229269  2306 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.229697  2306 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.230106  2306 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.235162  2289 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.098134  2323 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.147409   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.175482   928 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.227313   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.236891   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.237643   928 raft_consensus.cc:2368] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4.
I20260709 13:13:03.237785   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.178471   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.182318  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.198959   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.185897   995 rpcz_store.cc:276] 0709 13:13:01.932329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.932466 (+   137us) service_pool.cc:224] Handling call
0709 13:13:01.932722 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:02.638655 (+705933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.170012 (+531357us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54000474d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:03.175762 (+  5750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622295424}
I20260709 13:13:03.196812   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.240000   852 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:03.120751  2302 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.217132   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.239456   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.253146   930 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.250876   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.254619   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:03.240367   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 62023.3us
I20260709 13:13:03.187350   927 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:03.240034   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.189023  2319 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.192088   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.244166   846 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:03.256651   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.241809   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.260190   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.258574   819 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:03.257372   850 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:03.257985   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.201869   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.200922  2346 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:03.142087  2277 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.264675   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.196393  2345 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:13:03.146098   926 rpcz_store.cc:276] 0709 13:13:02.076450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.092856 (+ 16406us) service_pool.cc:224] Handling call
0709 13:13:03.143628 (+1050772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.144229 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.145379 (+  1150us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001f0750. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:03.145516 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397588352}
I20260709 13:13:03.272841   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.217815  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:37802 (request call id 295) took 1920 ms. Trace:
I20260709 13:13:03.223663  2348 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.274350   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.261549   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.262101   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.285461   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:03.212579   921 rpcz_store.cc:269] 0709 13:13:00.820088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.943054 (+122966us) service_pool.cc:224] Handling call
0709 13:13:03.128913 (+2185859us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54001f0754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:03.129066 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2199920512}
I20260709 13:13:03.366935  2289 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.287819  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.375949   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.288410  2120 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.377143   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.378307   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.289897  2317 consensus_queue.cc:1059] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:03.270556   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.270352   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:03.293414   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.383647   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:03.384358   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.294834   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.386263   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.387588   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.294204  2306 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.309859   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.252414  2310 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.243769  2041 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.404290  1004 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:03.376336  2289 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.272428  2336 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.289374  2302 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.257581  2260 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.223362  2365 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 13:13:03.236018  2211 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:13:03.305001  2323 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.388268   853 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:03.277841  2346 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.407742   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.480418   927 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.315979  2296 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.408618  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:03.409658   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 124425us
I20260709 13:13:03.487627   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:03.405831   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:03.488687   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82916.5us
I20260709 13:13:03.424597  2256 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.426574  1600 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.432264   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.494724   851 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:03.440192   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.336980  2319 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:03.405535  2339 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.491359   846 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.492197   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.364622  1595 rpcz_store.cc:276] 0709 13:13:01.280843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.286990 (+  6147us) service_pool.cc:224] Handling call
0709 13:13:03.196948 (+1909958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:03.430373  2335 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:03.503109   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 306311us
I20260709 13:13:03.504208   969 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.506091   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:03.508340   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 284449us
I20260709 13:13:03.489470  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.405174  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.495595  2274 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.513253   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.514333   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:03.496886  2339 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:03.526001   851 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:03.511059  2256 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.532510   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.532984   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.538232   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.538328   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.541828   852 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.523249  2345 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.567909  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.569928  1599 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:03.543229   852 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.529507  2339 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.572763  2213 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.522560  1000 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:03.534351  2129 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.529971  2194 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.541123   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.535852  2256 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:03.589676   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 91478.5us
I20260709 13:13:03.531670  2318 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:13:03.574836  2345 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:03.515663  2296 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.581948  2129 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:03.549909   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:03.577940  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 209655us
I20260709 13:13:03.619673   851 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.582626  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:03.616380  2345 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.618827  2296 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:03.616698  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.626976   848 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:03.620381  2129 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.541373  2211 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:13:03.642570  2296 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.655232  2373 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:13:03.658682  2279 consensus_meta.cc:287] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.815s	user 0.004s	sys 0.004s
W20260709 13:13:03.643155  2366 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:13:03.643344  1595 raft_consensus.cc:2393] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:13:03.643879  2368 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:13:03.644274  2367 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:13:03.663573   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 1215) took 1051 ms. Trace:
I20260709 13:13:03.669546   849 rpcz_store.cc:276] 0709 13:13:02.612422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.612537 (+   115us) service_pool.cc:224] Handling call
0709 13:13:03.663271 (+1050734us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540019e6d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:03.663479 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995532672}
I20260709 13:13:03.659088  2279 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.666239  2288 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.727980   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.647218  2041 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:03.648593  2300 raft_consensus.cc:3055] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:13:03.654513  2331 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.742s	user 0.007s	sys 0.000s
I20260709 13:13:03.730171  2288 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:03.666731  2296 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.667925  2274 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:03.733590   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 220383us
W20260709 13:13:03.644596  2369 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:13:03.735304   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 194313us
I20260709 13:13:03.735304   855 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.744503   849 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:03.739529  2300 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:03.769984  2300 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.771142  2300 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.729701  2289 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:03.732261   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 197749us
I20260709 13:13:03.625469  2306 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.783401   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.785416  1566 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.781895   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.764854  2310 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:03.752009   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 182285us
I20260709 13:13:03.788869   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.792616   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.793224   848 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.792903  2345 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.793460  2260 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.794584   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.794960  2345 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.795286  2260 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.745940  2331 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.761945  2339 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.796967  2382 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:03.797458  2339 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.764973  2052 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.798067  2052 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:03.798866  2052 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.767947  2296 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.799928  1000 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.799849  2052 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.754932  2288 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.744729  2375 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.810163  2375 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:03.810326  2288 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.811120  2375 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.812522  2375 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.774330  1595 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.775804  2365 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.775908  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:03.778049   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.784032   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.634965  2372 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.789113  2298 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.724615  2256 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.823909  2298 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:13:03.824695  2211 consensus_queue.cc:1175] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:13:03.790735  2213 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:03.789995  2279 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.824880  2298 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.825973  2279 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.826366  2279 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:03.827133  2373 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:03.827250  2279 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.827801   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.827746  2373 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.828598   849 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.786352  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.794870   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.830664  2342 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.831177  2342 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:03.832046  2342 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.832500  2294 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:03.833202  2346 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.833693  2294 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:03.817303  2345 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:03.834096  2294 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:03.834492  2345 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.795676   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.755915  2274 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.842890  2274 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.847325   852 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:03.654246  2129 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.848026   923 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:03.772877   969 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.807894   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.809163   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.800239  2378 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:13:03.816609   846 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.816084  2365 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:03.817289   921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:13:03.819597   854 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.818084  2372 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:03.858484  2129 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:03.858826  2372 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:03.748839  2357 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.004s
I20260709 13:13:03.828533  2279 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.791656   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.799778   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.819954   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:03.809064  2381 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:03.852183   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 44366.6us
I20260709 13:13:03.839985  2211 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:03.853860  2378 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:03.854732  2339 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.855695  2382 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:03.853116   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80306.2us
I20260709 13:13:03.858290  2312 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.861951   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.862998   850 raft_consensus.cc:2393] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:03.837553   851 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:03.863368   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 43597.8us
I20260709 13:13:03.864466   921 raft_consensus.cc:606] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:13:03.865034   925 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:03.879201  1004 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.835047  2294 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.865473  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.865864  2365 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.853884  2346 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.870373   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.888067   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:03.888672   997 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:03.889344   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.878684  2373 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.891266  2335 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.891534  2345 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.870604  2378 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:03.874886   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.858863   848 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:03.890168  2041 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:03.901865  2294 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.902083  2256 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.878845  2389 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:13:03.907096  2389 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:03.907475  2389 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:03.908655   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 55602.3us
I20260709 13:13:03.908331  2389 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:03.909696  2389 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:03.910172   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.917753  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:03.922320  2346 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.910456  2365 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.925086   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.925529  2331 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:03.925740   925 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.872798  2211 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.927800  2211 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:03.925555   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 1239) took 1016 ms. Trace:
W20260709 13:13:03.877840   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 78158.4us
I20260709 13:13:03.933677   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:03.934048   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.867318  2357 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.883037   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.883394   995 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.884840  1004 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.882269  2288 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.878638   921 raft_consensus.cc:614] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:03.868690  2129 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.886731   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:03.897575   853 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:03.898236   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.881510   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.880398  2312 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:03.931219   854 rpcz_store.cc:276] 0709 13:13:02.909261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.006280 (+ 97019us) service_pool.cc:224] Handling call
0709 13:13:03.925336 (+919056us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54001cbbd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:03.925477 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843462400}
I20260709 13:13:03.935801   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.937021   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.938308   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:03.944132   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:03.947789   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66369.9us
I20260709 13:13:03.948176   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.948607   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:03.949812   930 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:03.949880   924 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.951006  1593 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.951273  2371 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.940605  2211 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:13:03.950088   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.952682  2345 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.953944   925 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:03.953595  2373 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:03.954646  2373 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:03.955749   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:03.956455   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:03.946905  2129 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.964429  2129 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.967715  2339 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:03.967839   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:03.940912  2382 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.968346  2339 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:03.960943  2348 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.969381  2348 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:03.969764  2348 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:03.963922   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.935042  2277 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:03.975435   753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 220) took 3315 ms. Trace:
W20260709 13:13:03.988193  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45239.1us
I20260709 13:13:03.990837   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 228) took 3168 ms. Trace:
I20260709 13:13:03.992017   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.955955  2346 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.995378   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.962973  2345 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.996623   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:03.998685   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:03.989187   753 rpcz_store.cc:276] 0709 13:13:00.659983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.880206 (+220223us) service_pool.cc:224] Handling call
0709 13:13:03.975360 (+3095154us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:02.137971 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:02.269778 (+131807us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:02.273828 (+  4050us) write_op.cc:435] Acquired schema lock
0709 13:13:02.273854 (+    26us) tablet.cc:662] Decoding operations
0709 13:13:02.274459 (+   605us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:02.274508 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:02.274561 (+    53us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:13:02.289181 (+ 14620us) tablet.cc:701] Row locks acquired
0709 13:13:02.291554 (+  2373us) write_op.cc:260] PREPARE: finished
0709 13:13:02.331170 (+ 39616us) write_op.cc:270] Start()
0709 13:13:02.338465 (+  7295us) write_op.cc:276] Timestamp: P: 1783602782324116 usec, L: 0
0709 13:13:02.339754 (+  1289us) op_driver.cc:348] REPLICATION: starting
0709 13:13:02.417254 (+ 77500us) log.cc:844] Serialized 1308 byte log entry
0709 13:13:03.645155 (+1227901us) op_driver.cc:464] REPLICATION: finished
0709 13:13:03.748280 (+103125us) write_op.cc:301] APPLY: starting
0709 13:13:03.748349 (+    69us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:03.748476 (+   127us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:03.748611 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:03.776957 (+ 28346us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:03.777202 (+   245us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:13:03.777274 (+    72us) write_op.cc:312] APPLY: finished
0709 13:13:03.777526 (+   252us) log.cc:844] Serialized 35 byte log entry
0709 13:13:03.905649 (+128123us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b60000221c8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d064a9161 00007f0d06615815 00007f0d066177c4 00007f0d0658a473 00007f0d0658b8c6 00007f0d0658b32c 00007f0d06577389 00007f0d06576b53 00007f0d0657ad61 00007f0d0657ad19 00007f0d0657aca9
0709 13:13:03.905711 (+    62us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.905800 (+    89us) write_op.cc:454] Released schema lock
0709 13:13:03.905841 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:03.906128 (+   287us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.906157 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25129,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":102925,"apply.run_cpu_time_us":5334,"apply.run_wall_time_us":157841,"mutex_wait_us":2767,"num_ops":3,"prepare.queue_time_us":793537,"prepare.run_cpu_time_us":4161,"prepare.run_wall_time_us":306579,"replication_time_us":1282971,"spinlock_wait_cycles":11315072,"thread_start_us":86799,"threads_started":1}]]}
I20260709 13:13:03.970749  2114 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.008555   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:03.988261  2381 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.997131  2052 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.012425  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.016750   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.013078  2385 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:03.990947  2288 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.988723  2339 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:04.018296   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:04.015769  2129 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.033483   843 tablet_service.cc:1558] Processing DeleteTablet for tablet b8c347b6586f486192f48c46974a2526 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46352
I20260709 13:13:04.034760   929 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.023156   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.025982   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.036258   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.037437   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.038496   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:04.026650   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71020us
I20260709 13:13:04.039482   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.010552  2323 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.999346  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:04.040174   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.044528   995 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.048116   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 223) took 2983 ms. Trace:
I20260709 13:13:04.042021   895 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.014257  2381 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:04.019757  2385 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.996649  2394 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.054931  2388 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.047255  2289 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.059752  2211 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:04.023206  2386 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.040277  2327 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.019447  2305 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.073112   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:04.048673   751 rpcz_store.cc:276] 0709 13:13:01.064748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.064919 (+   171us) service_pool.cc:224] Handling call
0709 13:13:04.048041 (+2983122us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:02.441529 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:02.444250 (+  2721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:02.444279 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:13:02.444295 (+    16us) tablet.cc:662] Decoding operations
0709 13:13:02.444779 (+   484us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:02.444824 (+    45us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:02.444852 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:13:02.445103 (+   251us) tablet.cc:701] Row locks acquired
0709 13:13:02.445122 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:13:02.445255 (+   133us) write_op.cc:270] Start()
0709 13:13:02.445368 (+   113us) write_op.cc:276] Timestamp: P: 1783602782445205 usec, L: 0
0709 13:13:02.445385 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:13:02.522949 (+ 77564us) log.cc:844] Serialized 980 byte log entry
0709 13:13:03.768638 (+1245689us) op_driver.cc:464] REPLICATION: finished
0709 13:13:03.867145 (+ 98507us) write_op.cc:301] APPLY: starting
0709 13:13:03.870106 (+  2961us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:03.883980 (+ 13874us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:03.903724 (+ 19744us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:03.904104 (+   380us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:03.904295 (+   191us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:13:03.904346 (+    51us) write_op.cc:312] APPLY: finished
0709 13:13:03.904578 (+   232us) log.cc:844] Serialized 29 byte log entry
0709 13:13:03.904761 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.904843 (+    82us) write_op.cc:454] Released schema lock
0709 13:13:03.904882 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:03.905258 (+   376us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.905297 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27595,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":91933,"apply.run_cpu_time_us":2182,"apply.run_wall_time_us":38067,"num_ops":2,"prepare.queue_time_us":494475,"prepare.run_cpu_time_us":6668,"prepare.run_wall_time_us":169710,"replication_time_us":1323108,"thread_start_us":102444,"threads_started":1}]]}
I20260709 13:13:04.001957  2274 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:13:04.070194  2379 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:04.071316  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.072103   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66018.6us
I20260709 13:13:04.052119   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.054522  2345 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:04.044205  2300 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.045768  2339 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:04.067823  1596 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.015664  2303 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:13:04.051749   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:04.117460   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 65419.9us
I20260709 13:13:04.041591  2114 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:04.072592  2385 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.150282   928 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:04.151523   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:04.151921   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:04.039858  2393 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.069132  2381 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:04.134930  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:04.124063  2339 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:04.118691  2323 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.179589  2339 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.139240   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:04.184453   921 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.005205  2378 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.143213   925 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.188236  2378 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.144140  2303 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:04.191044  2303 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.144659  2375 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:04.193488  2375 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:04.194741  2375 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:04.195158  2375 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:04.096058   926 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:04.098873  2129 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:04.152113   855 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.198196   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:03.991240   749 rpcz_store.cc:276] 0709 13:13:00.822265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.825421 (+  3156us) service_pool.cc:224] Handling call
0709 13:13:03.990757 (+3165336us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:01.769982 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:01.770860 (+   878us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:01.770903 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:13:01.770929 (+    26us) tablet.cc:662] Decoding operations
0709 13:13:01.771356 (+   427us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:01.771432 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:01.771477 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:01.842473 (+ 70996us) tablet.cc:701] Row locks acquired
0709 13:13:01.842623 (+   150us) write_op.cc:260] PREPARE: finished
0709 13:13:01.877172 (+ 34549us) write_op.cc:270] Start()
0709 13:13:01.885389 (+  8217us) write_op.cc:276] Timestamp: P: 1783602781874163 usec, L: 0
0709 13:13:01.923534 (+ 38145us) op_driver.cc:348] REPLICATION: starting
0709 13:13:02.048838 (+125304us) log.cc:844] Serialized 533 byte log entry
0709 13:13:02.682798 (+633960us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b18003480a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:03.270809 (+588011us) op_driver.cc:464] REPLICATION: finished
0709 13:13:03.620178 (+349369us) write_op.cc:301] APPLY: starting
0709 13:13:03.628942 (+  8764us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:03.644729 (+ 15787us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3c32a 00007f0d02ac1f14 00007f0d02ac1ec3 00007f0d02ac1e38 00007f0d02ac1df3 00007f0d02ac10dc 00007f0d02aeae16 00007f0d02ad2cbb 00007f0d02abd39d 00007f0d02ac0d75
0709 13:13:03.702764 (+ 58035us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:03.722830 (+ 20066us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:03.725923 (+  3093us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:03.726136 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:03.731043 (+  4907us) write_op.cc:312] APPLY: finished
0709 13:13:03.772026 (+ 40983us) log.cc:844] Serialized 23 byte log entry
0709 13:13:03.786902 (+ 14876us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.787900 (+   998us) write_op.cc:454] Released schema lock
0709 13:13:03.804594 (+ 16694us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:03.840535 (+ 35941us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:03.849902 (+  9367us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1022,"reported_tablets":1,"spinlock_wait_cycles":1766272,"child_traces":[["op",{"apply.queue_time_us":263986,"apply.run_cpu_time_us":2885,"apply.run_wall_time_us":260362,"num_ops":1,"prepare.queue_time_us":447713,"prepare.run_cpu_time_us":13379,"prepare.run_wall_time_us":339399,"raft.queue_time_us":847686,"raft.run_cpu_time_us":49348,"raft.run_wall_time_us":623719,"replication_time_us":1339198,"spinlock_wait_cycles":27764352,"thread_start_us":449954,"threads_started":4,"wal-append.queue_time_us":53469}]]}
I20260709 13:13:04.097903   847 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:04.155400   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:04.156847   852 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.154445   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:04.149008   854 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.211254  2342 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:04.216722  2303 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.218814  2303 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:04.221132   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.230082   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:04.238006  2375 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.145845  2345 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.258515  2380 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.187s
I20260709 13:13:04.246243   967 leader_election.cc:400] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4.
I20260709 13:13:04.259526   967 leader_election.cc:403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:04.115788  2310 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.105317  2211 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:04.267763   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 115318us
I20260709 13:13:04.268692   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.268940   995 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:04.270200   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.247438   851 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20260709 13:13:04.071815  2318 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:13:04.094719  2395 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.132457  2114 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:04.040587  2373 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.247825  2289 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:13:04.247218   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 94452.9us
I20260709 13:13:04.282079   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.283161   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:04.098202  2372 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.097108  2260 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.150s
I20260709 13:13:04.253968  2346 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.283735   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:04.273373   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.044552  2312 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.272257  2211 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:04.279114   922 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:04.321796   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.281378  2114 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.278462  2310 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:04.326781  1003 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:04.324930   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.284179   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:04.259779  2394 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:04.284646  1596 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:04.279386  2300 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 13:13:04.338160   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55064.7us
I20260709 13:13:04.322603  2365 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4.
W20260709 13:13:04.340065   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56404.3us
I20260709 13:13:04.341617   921 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:04.341686   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.346591   924 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:04.294163  2336 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:13:04.326120  2256 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.327663  2349 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:04.329905  2395 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.331802  2396 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:04.336944  2373 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.377209  2289 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.372835  2211 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.359025  2288 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:04.381310  2288 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:04.381703  2288 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:04.349658  2052 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.380892  2211 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.359324  2318 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.384004  2318 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.384845  2318 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:04.363225  2392 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 13:13:04.337036  2375 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:04.391482   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:04.393435   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:04.399847   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 129739us
I20260709 13:13:04.379266  2396 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:04.400841   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.407001  1596 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:13:04.409015  2395 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:04.409466  2395 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.409298  2260 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:04.409808  2395 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.411445   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.414568  2394 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:04.417867   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76267.6us
I20260709 13:13:04.414976   967 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.422565   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.399466  2331 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.400482  2300 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.429471   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:04.399088  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53121.3us
I20260709 13:13:04.433079   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.435386  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.436018  2327 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:04.436637  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.436975  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:04.437646   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.438273   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.439006   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.427246  2401 tablet_replica.cc:333] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:13:04.454137   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:04.431782  2114 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.455971  1003 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.465050   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.431879  2325 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.431293  2399 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.475457   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:04.476953   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.447971  2331 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:04.472064   969 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.481603   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.452414  2300 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:04.452878  2327 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.459628  2404 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:04.500751   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 69863.5us
I20260709 13:13:04.489591  2296 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:04.491788  2397 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:13:04.492249  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.510855  1002 raft_consensus.cc:2393] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:04.427160  2310 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.505112  1004 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:04.524636   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.474069  2236 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.474258  2378 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:04.481155  2306 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.481267  2384 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:04.527575   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46062us
I20260709 13:13:04.426331  2396 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.534147   851 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:13:04.536584   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56319.1us
I20260709 13:13:04.539351   998 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:04.545663   930 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:13:04.549994   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 269007us
I20260709 13:13:04.553141   999 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:04.552292   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80314.1us
I20260709 13:13:04.521044  2388 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.528662  2378 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.532600  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.576575  1002 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.541644  2362 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:04.573844   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:04.579414   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.580216   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.554046  2399 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.584270  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.557425  2396 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:04.558805  2298 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:13:04.595778   851 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.575585  2378 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:04.605252  2416 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:04.555955  2342 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:13:04.578377  2339 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.584959  2388 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.579281  2362 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:04.610037  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.616185  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.617674  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.610806   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:04.613389   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.613893   850 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:04.620005  2211 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:04.620548  2211 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.621124  2395 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.622498  2277 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.623183  2277 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.623751  2277 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.625273  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.559866  2325 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:13:04.630980  2412 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:04.630640   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 97126.7us
I20260709 13:13:04.603416  2194 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:04.633771   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:04.634275   995 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:04.636545   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:04.670009   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.559056  2327 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:04.690750   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57335.9us
W20260709 13:13:04.694443  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65170.9us
W20260709 13:13:04.637332  2389 consensus_meta.cc:287] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.002s
I20260709 13:13:04.693286   997 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:04.628721  2388 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.701575  1003 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:04.701855   998 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:04.702446   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.705497  2411 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:04.712620   997 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:04.716751   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:04.697036   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63337.5us
I20260709 13:13:04.638172  2298 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:04.725514  2404 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.595485  2331 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.702945  2274 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:13:04.741989  2274 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:04.742692  2274 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.746389   821 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:04.729338   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.597541  2300 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.637044  2306 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:13:04.639532  2289 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.697242  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.756309   997 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:04.740141  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:04.770623  2406 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:13:04.688530  2194 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.750388  2296 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.754849  2236 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:04.781406  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 76709.5us
W20260709 13:13:04.779701   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 82217.6us
I20260709 13:13:04.731539  2114 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:04.784623   851 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.785602   850 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.788993   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:04.804368   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:04.806905   999 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:04.757352  2407 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.771243  1000 raft_consensus.cc:2376] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2. Re-sending same reply.
I20260709 13:13:04.823877   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:04.845682   999 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:04.846371  2298 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:04.778609  2404 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:04.796260  2279 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.000s
I20260709 13:13:04.749442  2362 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:04.862041  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:04.842623  2289 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:13:04.862722   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58437.8us
I20260709 13:13:04.856737   999 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:04.799237  2296 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:04.872172   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:04.848989  2357 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:04.877091  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:04.881902   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:04.808000  2394 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:04.888906   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.850667  2395 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:04.884057   999 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:04.885154   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:04.866829  2386 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.918413   850 raft_consensus.cc:2393] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:04.918460   851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:04.917263  1594 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:04.919279   851 raft_consensus.cc:606] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:13:04.919631  2377 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:04.919791   851 raft_consensus.cc:614] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:04.920068  2377 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.921005  2377 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.808390  2375 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:13:04.929695  2430 raft_consensus.cc:2764] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:13:04.872705  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 152215us
I20260709 13:13:04.880409  2339 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:04.950881  2041 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:04.890673  2318 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:04.891852  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:04.961972   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.924516  2402 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:13:04.973459  2380 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:04.975751  2318 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:04.960351   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:05.007498   850 raft_consensus.cc:606] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
W20260709 13:13:04.980733   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.869220  2404 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.975544  2327 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:05.001075   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.960165  2377 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.003556   853 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:04.849515  2414 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
W20260709 13:13:04.963148  2385 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.692s	user 0.006s	sys 0.004s
I20260709 13:13:04.862706  2388 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:04.891484  2289 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:05.028401   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.028934   999 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.916548  2394 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.004221  2380 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:05.013569   922 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.678s	user 0.007s	sys 0.003s
I20260709 13:13:05.034371   924 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:05.012418  2310 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.059082   851 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.028077  2298 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.052194   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.012367  2426 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:13:05.091499   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:05.107090   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 54999.5us
I20260709 13:13:05.111815   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.112360   999 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:13:05.113444   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.114553   855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:05.115116   855 raft_consensus.cc:606] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:05.120218  2380 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:05.129215  1567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 2.173s (SENT) [suppressed 64 similar messages]
I20260709 13:13:04.999774  2296 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.094489  2194 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.039659   922 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:05.111630  2339 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:05.128008  2417 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:13:05.028206  2385 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.150724  2431 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:05.151275  2431 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.152381  2289 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.153491  2431 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.153179   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1247) took 1003 ms. Trace:
I20260709 13:13:05.155529   853 rpcz_store.cc:276] 0709 13:13:04.149919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.150045 (+   126us) service_pool.cc:224] Handling call
0709 13:13:04.150330 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:05.015394 (+865064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.152945 (+137551us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400409cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.153125 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961261952}
I20260709 13:13:05.157922   749 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:05.138008  2348 consensus_meta.cc:287] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.168s	user 0.008s	sys 0.005s
I20260709 13:13:05.036514  2427 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:13:05.160708   849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:05.178779   849 raft_consensus.cc:606] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:05.147218  2041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:46352 (request call id 71) took 1016 ms. Trace:
I20260709 13:13:05.158959  2399 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:13:05.164443   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.165566  1593 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:05.162055   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:05.174693   749 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.145279  2432 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:13:05.147080  2418 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:13:05.142225  2312 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.123513  2414 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:13:05.106802  2310 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.179805  2041 rpcz_store.cc:276] 0709 13:13:04.113964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.114131 (+   167us) service_pool.cc:224] Handling call
0709 13:13:04.228357 (+114226us) log.cc:844] Serialized 494 byte log entry
0709 13:13:05.130113 (+901756us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15546,"raft.queue_time_us":17116,"raft.run_cpu_time_us":761,"raft.run_wall_time_us":62493,"spinlock_wait_cycles":31710080,"thread_start_us":175241,"threads_started":1,"wal-append.queue_time_us":258743}
I20260709 13:13:05.186028  1593 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:05.189333  2399 consensus_queue.cc:1243] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:05.186458   999 raft_consensus.cc:2376] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2. Re-sending same reply.
I20260709 13:13:05.193485   930 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:05.197511 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:05.201849  2404 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.202226 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:05.198467   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.204027   853 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
W20260709 13:13:05.196082  2434 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:13:05.128623  2423 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:13:05.179864  2335 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.176879   921 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:05.192111  2346 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.208523   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.206704   749 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:05.209446  2346 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:05.210383  2346 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:05.200747  2381 consensus_meta.cc:287] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.929s	user 0.003s	sys 0.008s
W20260709 13:13:05.216470   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 91422.3us
I20260709 13:13:05.184504  2348 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.147387  2318 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.215994   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7de58b0d5ad94d869fac3b0491143543 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:13:05.225936  2288 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.844s	user 0.007s	sys 0.004s
I20260709 13:13:05.217190  2429 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:05.249382  1593 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.247372  2288 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.250789  1593 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.252180  2288 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:05.253444  2303 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.032s	user 0.012s	sys 0.000s
I20260709 13:13:05.232668  2346 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.254698  2414 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.255203  2414 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.260028  1001 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:05.261191   928 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:05.262369   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:05.257551  2414 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:05.269050  2373 consensus_meta.cc:287] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.877s	user 0.010s	sys 0.000s
I20260709 13:13:05.270076  2399 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.270622  2399 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.275681  2211 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.226949  2120 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.280359   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.280643   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:05.281111   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:05.283154  1567 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.284440   849 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.282056   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.289825   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:05.294950   821 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.297492   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.300542   851 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:05.300705  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.302418   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:05.304401   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.307152   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.309751  2409 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.286662  2120 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:05.307214  2421 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.309813   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:05.315155  2349 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.936s	user 0.002s	sys 0.009s
I20260709 13:13:05.227531  2381 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.311794  2426 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:05.311448  2373 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.304015  2213 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.314440  2423 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:05.309329  2211 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:13:05.314481   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.307726  1594 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:05.320963  2194 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:13:05.337931   996 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.325109   848 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:05.319332  2294 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:13:05.330109  2349 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.218521  2335 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.331986   847 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
W20260709 13:13:05.347389   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.352605   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:05.337572  2426 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:05.300197  2429 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.335282  2120 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.360504   852 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:05.344705  2423 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:05.326969  2331 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:05.358254  2213 raft_consensus.cc:993] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:05.361888   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.364655   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.364882   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.365674  2372 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.365990   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.366286  2373 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.367110  2274 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.324937  2298 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:05.360172  2412 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.000s
I20260709 13:13:05.368486  2412 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.368299  2298 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.368872   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.370040  2385 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.329806  2422 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:13:05.370448  2373 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.350673  2421 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:05.367283   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.373505   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:05.375379   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:04.556231  2345 consensus_queue.cc:1175] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64f2cb5b0e174002811ef197f9eec829" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:05.379274  2429 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.379526  1593 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:13:05.381249  1593 pending_rounds.cc:85] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 4
I20260709 13:13:05.382411  2436 raft_consensus.cc:993] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:05.382544  1593 pending_rounds.cc:107] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:13:05.383028  1593 raft_consensus.cc:2889] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:05.373656  2296 raft_consensus.cc:1081] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:05.383770  2310 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:05.384531  2378 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.002s
I20260709 13:13:05.386250   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.386873   924 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:05.384984  2378 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.390270   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.391862   924 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:05.370787  2298 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.399483  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:05.399889   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.400564   998 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:05.381369  2389 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.382982  2375 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.371631   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.399951  2310 raft_consensus.cc:1081] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:05.376724  2421 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.405406   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.403415  2375 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:05.377228  2346 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:13:05.407500  1004 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:05.411458   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:05.403151  2389 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.390904  2348 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.408001  2423 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.402379  2372 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.402719  2422 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.419106  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:05.421733  2398 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.413508  2382 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.424355   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.432096   999 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:13:05.419415  2389 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:05.430148   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.426263   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.429280  2422 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.406915  2426 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.420595   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.456104   925 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:05.457484   925 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.464005   999 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:05.464444   999 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.431574  2298 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:05.453709  2429 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.468130  2429 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.470256  2429 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.460654  2260 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:05.472312  2260 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.474568  2260 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.475988  2378 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.476775  2401 raft_consensus.cc:2243] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.447643  2414 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.482781  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:05.480481   925 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:05.484402  2399 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.485174  2421 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:05.486590   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.430460  2372 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.490829  2335 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:05.492861  2435 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.405s	user 0.002s	sys 0.001s
I20260709 13:13:05.497551   926 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.499290   926 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:05.500991   926 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:05.485987  2120 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.478578  2298 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.508692  2298 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:05.481045  2401 raft_consensus.cc:2272] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.511435  2422 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.512020   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.512708   997 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:05.513629   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.513809   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:05.514492   997 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:05.513671  2372 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.521481   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.529714   997 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:05.530893   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.531925   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:05.557510   999 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.516798  2422 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:05.565424   854 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:05.529327  2327 consensus_meta.cc:287] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.002s
I20260709 13:13:05.567232  2422 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.570955  2422 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:05.547940  2120 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.543408  2335 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.582481  2120 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.581743  2421 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.586267   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.588531  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:05.572526  2388 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:13:05.603761   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.596596  2120 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.610874  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.614935   998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:13:05.615513   998 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:13:05.615721   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.615921   998 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.616829  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.617340  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.617561  1593 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:05.622677  2401 ts_tablet_manager.cc:1916] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:05.623716   926 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:05.588299  2335 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.626155  2335 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.628481   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:05.632292   923 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:05.637840   819 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:05.639082  2339 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:05.639518  2339 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:05.639850  2306 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.641001   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 87) took 1208 ms. Trace:
I20260709 13:13:05.641386   995 rpcz_store.cc:276] 0709 13:13:04.432892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.434005 (+  1113us) service_pool.cc:224] Handling call
0709 13:13:04.434301 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.638493 (+204192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.640785 (+1002292us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000889d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.640947 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459471616}
I20260709 13:13:05.642588   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:05.643064  2410 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:13:05.643167   995 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:05.645068   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:05.640337  2306 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.601053  2421 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:05.649940   819 raft_consensus.cc:2686] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:05.652132   921 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:05.653662  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.654301  1600 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:05.649451  2306 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.656008  2420 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.656455  2420 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.656814  2420 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.636281  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.680462   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:05.684837   998 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:05.694711   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.693349  2120 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.697857   998 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.704496   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.705382   998 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:05.676077  2388 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.708722   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:05.711503   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:05.713357   998 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:05.713765   998 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:05.714756   998 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.716109   998 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.716391   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1258) took 1273 ms. Trace:
I20260709 13:13:05.715463   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1224) took 1701 ms. Trace:
I20260709 13:13:05.716768   854 rpcz_store.cc:276] 0709 13:13:04.442698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.442823 (+   125us) service_pool.cc:224] Handling call
0709 13:13:04.443109 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.567313 (+1124204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.716174 (+148861us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001cc5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.716335 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2580450048}
I20260709 13:13:05.719722   996 rpcz_store.cc:276] 0709 13:13:04.014445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.014676 (+   231us) service_pool.cc:224] Handling call
0709 13:13:04.016683 (+  2007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.344217 (+1327534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.715270 (+371053us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540034e9d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.715415 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3038043392}
I20260709 13:13:05.709510  2318 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.721835   996 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.721738  2318 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.650131  2339 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.704361  2377 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.706971  2362 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:05.731434   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.738680   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.739195  2422 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.740262   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.740939   925 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:05.741804   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.752111  2408 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.752583  2408 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:05.753204   749 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.025s	sys 0.019s
W20260709 13:13:05.753532   749 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.025s	sys 0.019s
I20260709 13:13:05.726164  2373 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.757019  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 95) took 1319 ms. Trace:
I20260709 13:13:05.757393   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.758606   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.757437  1001 rpcz_store.cc:276] 0709 13:13:04.437768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.440284 (+  2516us) service_pool.cc:224] Handling call
0709 13:13:04.440540 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.260578 (+820038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.756819 (+496241us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b5400047ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.756968 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884052224}
I20260709 13:13:05.760288   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.761163   921 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:05.762012   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.762763   921 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.746627  2339 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.766501  2120 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.770460  2425 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.719688  2388 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.772166   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.726909  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.780682  2423 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:05.780804  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 71944.9us
I20260709 13:13:05.776437   998 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:05.777138   997 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.744212  2377 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.786562   998 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:05.778033  2409 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.779177  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:05.779893   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.784992   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:05.761392   925 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:05.763751  2432 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.771015  2425 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:05.789917   922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1291) took 1797 ms (client timeout 1824 ms). Trace:
I20260709 13:13:05.790796   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:05.792840   929 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:05.794977   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:05.795831   922 rpcz_store.cc:269] 0709 13:13:03.996926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.004728 (+  7802us) service_pool.cc:224] Handling call
0709 13:13:05.789687 (+1784959us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540028ffd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:05.789821 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322703232}
I20260709 13:13:05.788924  2409 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:05.795892  2432 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.763945  2373 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:05.794252  2395 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:13:05.804464   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 82) took 1169 ms. Trace:
I20260709 13:13:05.788393  2412 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.807725  2432 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.796808  2385 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.804860   930 rpcz_store.cc:276] 0709 13:13:04.635013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.635130 (+   117us) service_pool.cc:224] Handling call
0709 13:13:04.635438 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:05.200533 (+565095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.804234 (+603701us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400082854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.804408 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280324352}
I20260709 13:13:05.812212   930 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.781894  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:05.815738  2449 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
I20260709 13:13:05.809875   921 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:05.822568   930 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.825541  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.827051  2428 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.827147  2389 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.827662  2428 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:05.828121  2428 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.816082  2408 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.833225  1003 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:05.836068   999 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.836604  2389 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:05.837301  2349 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:05.831429   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:05.840018   929 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:05.840082   997 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:05.856778  1002 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.860025  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 99) took 1421 ms. Trace:
I20260709 13:13:05.804477  2312 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:05.830523   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1259) took 1385 ms. Trace:
I20260709 13:13:05.860462  1004 rpcz_store.cc:276] 0709 13:13:04.438893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.525087 (+ 86194us) service_pool.cc:224] Handling call
0709 13:13:04.525388 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.414610 (+889222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.859803 (+445193us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54000898d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.859966 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2028179328}
I20260709 13:13:05.834545  2327 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.802268  2409 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.865164  2424 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.828967  2423 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:13:05.862255  2408 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.860689  2362 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.864612  2422 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.881795   854 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:05.884838   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.820971  2412 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.868417  2318 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.887712  2389 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.866081   848 rpcz_store.cc:276] 0709 13:13:04.443358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.443552 (+   194us) service_pool.cc:224] Handling call
0709 13:13:04.443912 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.348073 (+904161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.810253 (+462180us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b540040a1d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:05.828852 (+ 18599us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2007292800}
I20260709 13:13:05.888181   926 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:05.888698  2306 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.891460   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.911141   924 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.892592   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:05.912789   927 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.869035  2327 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:05.901382  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:05.865159  2298 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.898427  2300 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.896205   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:05.923810   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:05.941659   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.912935  2377 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:05.886639  2362 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.886564  2421 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:05.898909  2420 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.945887   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:05.930694  2395 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.957507  2410 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:05.942189  2396 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.958467  2378 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:05.944834  2372 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:13:05.959760  2348 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:13:05.984155   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.911723  2389 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.021898  2377 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.022550  2377 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:05.918814  2425 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:05.990808   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45002.5us
I20260709 13:13:05.988790  1595 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:05.987360  2373 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.026003  1001 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:05.930308  2300 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.037784   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:05.991621  2410 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:06.025178  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.023342  2362 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.039884  1004 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:05.978310  2427 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:13:05.996718  2399 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:06.041841   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:06.045670   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:06.048547  2412 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.055289  2372 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:06.054535  2396 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:06.049459  2373 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:06.064983  2399 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:06.094134  2300 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.094139  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:06.085143  2388 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.089128  2373 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:06.091449  2396 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:06.081189  1592 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:06.104647  2298 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.120127  2428 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.123047  2428 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.124351  2414 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.124869  2414 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:06.125865  2414 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.114919  2420 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.110226  2306 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.127362  2414 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.113631  2394 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.111244  2377 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.114478  2389 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.084666  2410 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:06.175577   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 417580us
W20260709 13:13:06.180248   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 453413us
I20260709 13:13:06.183810   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.185563   848 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:06.050483   926 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:06.191634   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:06.203907   928 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.004s	sys 0.006s
I20260709 13:13:06.203955 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:06.204774   928 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.006s
I20260709 13:13:06.206168   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:06.208223   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:06.209359   851 raft_consensus.cc:2368] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3.
I20260709 13:13:06.209748   995 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:13:06.210242   995 pending_rounds.cc:85] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 1
I20260709 13:13:06.210798   995 pending_rounds.cc:107] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 13:13:06.213326  1600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.037s
W20260709 13:13:06.213898  1600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.037s
I20260709 13:13:06.211272   995 raft_consensus.cc:2889] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:06.115116  2399 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.244796  2260 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:06.252765  2310 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.253573  2310 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.254504  1566 leader_election.cc:400] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3.
I20260709 13:13:06.254995  1566 leader_election.cc:403] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:06.255784  2396 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.259884   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:06.271134  2410 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.274526  2339 raft_consensus.cc:476] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Not starting pre-election -- already a leader
W20260709 13:13:06.277199  2417 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:13:06.281256  2372 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:06.282294  2372 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.283948  2300 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:06.289196   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:06.291563  2335 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.297186  2213 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.297765  2213 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:06.300989  2373 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.322396  2335 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.299403  2213 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.337642   928 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:06.341877   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 294581us
I20260709 13:13:06.345860  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:06.347781  2427 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.352463  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:06.353415  1593 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:06.354830   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:06.356355  2461 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.357187  2461 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.358289  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:06.334302  2310 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.359347   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:06.359756  2335 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.361121   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:06.361897   921 raft_consensus.cc:2376] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3. Re-sending same reply.
I20260709 13:13:06.362670  2335 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:06.364162   923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1208) took 1325 ms (client timeout 1535 ms). Trace:
W20260709 13:13:06.364622   923 rpcz_store.cc:269] 0709 13:13:05.038503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.038646 (+   143us) service_pool.cc:224] Handling call
0709 13:13:06.363729 (+1325083us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54003287d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:06.363920 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932264320}
I20260709 13:13:06.367059  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.367431  2310 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:06.368216   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.368943  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:06.341763   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47159.6us
I20260709 13:13:06.374167  1002 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:06.376603  2372 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:06.377350   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:06.382278  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:06.375020  2461 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.384167  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.385349   997 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:06.385617   847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.009s	sys 0.010s
W20260709 13:13:06.386008   847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.009s	sys 0.010s
I20260709 13:13:06.387193  2422 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:06.387543   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:06.388020  2422 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.389674  2260 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.390738  2422 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.391000   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:06.378635   925 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.393864  2300 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.394384  2300 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.394815  2300 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:06.395140  1592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 336) took 1690 ms (client timeout 1765 ms). Trace:
I20260709 13:13:06.400545  1600 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.368263  2310 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.384236   929 raft_consensus.cc:2955] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:06.397683   922 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:06.395617  1592 rpcz_store.cc:269] 0709 13:13:04.704399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.731564 (+ 27165us) service_pool.cc:224] Handling call
0709 13:13:06.394781 (+1663217us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004c9d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:06.394958 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416553728}
I20260709 13:13:06.399235  2213 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.415845  2335 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:06.415855   927 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.004s
I20260709 13:13:06.411649  2327 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.423504   927 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:06.425187  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:06.425026  2335 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.426045  1002 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.428385   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.429049  2380 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.431900  2381 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.434610  1002 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:06.451972   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.452711   846 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
W20260709 13:13:06.456955  2473 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:13:06.465830  1600 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:06.470480  1002 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:06.455327  2335 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.469779   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:06.477895  2469 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.091s	user 0.004s	sys 0.000s
I20260709 13:13:06.479359  2401 ts_tablet_manager.cc:1929] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:13:06.537915  2401 log.cc:1199] T b8c347b6586f486192f48c46974a2526 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/b8c347b6586f486192f48c46974a2526
I20260709 13:13:06.537768   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.538760   850 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:06.539731  2401 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46352 (request call id 70) took 2533 ms. Trace:
I20260709 13:13:06.540207  2401 rpcz_store.cc:276] 0709 13:13:04.015070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.033143 (+ 18073us) service_pool.cc:224] Handling call
0709 13:13:04.373399 (+340256us) ts_tablet_manager.cc:1151] Deleting tablet b8c347b6586f486192f48c46974a2526
0709 13:13:04.394966 (+ 21567us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:13:06.132065 (+1737099us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:06.479073 (+347008us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:06.539630 (+ 60557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":948139,"tablet-delete.queue_time_us":301409,"thread_start_us":240850,"threads_started":1}
I20260709 13:13:06.469339  2260 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.542712   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.547000   998 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:06.494642  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:06.479655  2052 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.477741  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:06.571034   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet b8c347b6586f486192f48c46974a2526 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:13:06.608502  1003 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.563720   996 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:06.574079  2331 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:06.607936  2411 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.611740  2414 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.612198  2411 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.612578  2414 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.615012  2414 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.494431  2296 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:13:06.615996   849 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:06.615713  2423 consensus_queue.cc:1059] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:13:06.616916  2423 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:06.617167   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.608325  2424 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.618227   854 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.618379   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.618714  2424 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.619314   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.621600  2424 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.622280  2325 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.622830  2325 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.625181  2325 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.625275   925 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:06.628193  2300 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.630762  2300 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.633277   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:06.617290  2423 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:06.567023   855 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:06.569640  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.562280   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.569067  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:06.615024  2411 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.633431   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1296) took 1059 ms. Trace:
W20260709 13:13:06.639480   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56842.3us
I20260709 13:13:06.637913   819 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:06.472290  2310 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.640336   995 rpcz_store.cc:276] 0709 13:13:05.573586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.645862 (+ 72276us) service_pool.cc:224] Handling call
0709 13:13:05.646098 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.247860 (+601762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.633122 (+385262us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b540035b954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:06.633348 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1293789952}
I20260709 13:13:06.643049   852 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:06.567788  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:06.643227   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.645259  2296 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:06.645606   922 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:06.645794   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70170.2us
W20260709 13:13:06.565580  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.576676   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:06.559549   997 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.647893  2298 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:06.648116   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 86947.7us
W20260709 13:13:06.676505   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.648655  2422 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:06.651266   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:06.562453   924 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.006s
I20260709 13:13:06.659126   998 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:06.659795  2423 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.562379   848 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:06.570365  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:06.671707   999 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:06.676617   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:06.647939  2306 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:13:06.659062  2362 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:06.482286  2375 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.683444  1001 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.683383   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.651276  2427 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3.
I20260709 13:13:06.661497  2296 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:06.688889  1595 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:06.561254  2052 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:06.653836  2414 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.689626   924 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:06.687978   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:06.691444   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.698266   849 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:06.696641  1002 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.704425   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.708139  1000 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.708849   855 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.707985   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.689561  2422 raft_consensus.cc:697] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:06.709334   855 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:06.689956  2362 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:06.715844   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.716449   999 raft_consensus.cc:2410] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:06.725237   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:06.729946   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.725741   998 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:06.739535   854 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:06.739854   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.741237  2432 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:06.741575   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:06.743744  2432 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.744246  2432 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.744534  2480 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:06.744645  2432 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.745172  2480 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:06.740581  2052 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.747308  2052 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.747078  2362 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.750303  2420 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:06.750976  2420 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.751690   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:06.706017  2213 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:13:06.757839  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:06.758994   854 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:06.764986   850 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:06.765599   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 162408us
I20260709 13:13:06.760177  2296 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.760387  2422 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.773928   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:06.728232  2375 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.777915   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:06.772553   818 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:06.783107  2298 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:06.784686  2461 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.786327   848 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:06.798727  2461 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:06.786502  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50570.5us
I20260709 13:13:06.758620   855 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.802491  1001 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:06.796684   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 80) took 1143 ms. Trace:
W20260709 13:13:06.760409  2395 consensus_meta.cc:287] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.011s
I20260709 13:13:06.805941  1002 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:06.813580   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55972.3us
W20260709 13:13:06.814967   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.814376   846 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
W20260709 13:13:06.788547  2410 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.005s
I20260709 13:13:06.828163  2423 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.832468  2395 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:06.833786   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 197391us
W20260709 13:13:06.834528   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.835158   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:06.835731  2486 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:06.837281  2373 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:06.837805  2373 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.838235  2373 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.735412  2414 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.823246  2380 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.803227   853 rpcz_store.cc:276] 0709 13:13:05.655076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.655197 (+   121us) service_pool.cc:224] Handling call
0709 13:13:05.655700 (+   503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:05.687908 (+ 32208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.687941 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:05.691060 (+  3119us) log.cc:844] Serialized 520 byte log entry
0709 13:13:06.184655 (+493595us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:06.184833 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.756461 (+571628us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b54003aff54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:06.756502 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.756558 (+    56us) raft_consensus.cc:1685] finished
0709 13:13:06.756569 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.796469 (+ 39900us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9745,"spinlock_wait_cycles":72782208,"thread_start_us":477593,"threads_started":1,"wal-append.queue_time_us":443389}
I20260709 13:13:06.850406  2380 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:06.847343   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 89466.6us
I20260709 13:13:06.848760  2410 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:06.848907  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:06.849577   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:06.854557   924 raft_consensus.cc:2368] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3.
I20260709 13:13:06.855304   995 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:06.844123  2335 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:06.843580  2424 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.858170  1000 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:06.857867  2395 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.859082   855 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.861091   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:06.862365  2335 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:06.863685  2432 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.866329  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:06.855326  2380 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.865871  2395 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.871498   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 81) took 1212 ms. Trace:
I20260709 13:13:06.878547   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:06.896401  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.880162  2395 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.882547  2410 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:06.905251   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:06.896806  2052 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:13:06.916761   855 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:06.908567  2410 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.919313  2389 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:06.920447   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:06.919809   853 raft_consensus.cc:2955] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:06.887830  2414 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:06.887123  2380 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:06.908298  2298 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:06.900875  2424 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.901850  2420 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:06.954717  2373 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:06.917568  2422 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.932257   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:06.905048  2411 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.923429  2389 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.921595  2429 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:06.925288  2410 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.923756  2306 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:06.959906   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 99057us
I20260709 13:13:06.961922  2414 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:06.973117  2327 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.921420  2325 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:13:06.977744  1001 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:06.979197  1000 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:06.958907  2298 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:06.984861   896 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:06.963825  2389 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.974838  2327 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.010162  2327 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.008646   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.011626   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.003446  2484 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:13:07.013113   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.014459  2380 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:07.014894  2380 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.015236  2380 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.999384  2306 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.017979  2306 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.019348  2424 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.003593  2481 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:13:06.992722  2414 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:07.020726   997 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.022007   848 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.021690  2414 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.024996   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.026317   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.027065   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.027940  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.028290  2310 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.028698  1000 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.029003  2461 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.028872  2310 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.029698  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.030478  1000 raft_consensus.cc:2393] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.030292  2461 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:07.030627  2432 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:07.031050  2310 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.031214  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.050323  1000 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:07.052201  2490 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:07.052677  2373 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:07.055867   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.007994  2410 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.981848  2335 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.008625   924 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.008939   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.009284   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.032218   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.032433  2362 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.061625   996 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.063939  2306 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:07.032934   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.034385  2480 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:06.991322  2429 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:07.007994   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.097390  2399 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.097919  2399 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.084249  2373 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.100342  2373 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.036666   849 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:07.037343   848 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:07.027078   850 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:07.038326  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.005805   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:06.975942  2422 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.039917   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:07.103190   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 63359us
I20260709 13:13:07.039968   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.104019   995 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.106015   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:07.106262  2420 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.106837  2420 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:07.095341  2327 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.041174  2298 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.107815  2420 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.048048   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:07.048703   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.049350  2375 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:07.007234   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:07.056751   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49216us
I20260709 13:13:07.079946   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.035264  2296 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.084256   999 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.056399  2380 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:07.085804   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 85363.8us
W20260709 13:13:07.090006  2489 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:13:06.880697   847 rpcz_store.cc:276] 0709 13:13:05.655415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.655852 (+   437us) service_pool.cc:224] Handling call
0709 13:13:05.656215 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:05.791726 (+135511us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0cffa350ac 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa4f455 00007f0d02b37060
0709 13:13:06.386633 (+594907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.386694 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:06.504215 (+117521us) log.cc:844] Serialized 26 byte log entry
0709 13:13:06.708455 (+204240us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:06.714962 (+  6507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.731210 (+ 16248us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.840594 (+109384us) raft_consensus.cc:1685] finished
0709 13:13:06.842983 (+  2389us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.867344 (+ 24361us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5118,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":3620736,"thread_start_us":793606,"threads_started":2,"wal-append.queue_time_us":283660}
I20260709 13:13:07.100608  2389 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.107520  2041 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:13:07.041124  2395 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.036041  2409 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.112637   927 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:07.080523  2325 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.115686  2373 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:07.140280   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 91668.4us
W20260709 13:13:07.147696   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 67840.1us
I20260709 13:13:07.153390   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:07.140765   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:07.148453   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 172690us
I20260709 13:13:07.149578  2306 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:07.144870   929 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3.
W20260709 13:13:07.160801   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.178849   998 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.139556   925 raft_consensus.cc:2376] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2. Re-sending same reply.
I20260709 13:13:07.152922  2411 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.180660   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:07.160086  2306 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.165540  2480 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.137351  2429 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.166534  2375 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:07.165365  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:07.193022  2041 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.193764  2410 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.193914  2345 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:07.179239  2432 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:07.189960   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53804.6us
I20260709 13:13:07.164285   926 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:07.192816  2306 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.206827  2041 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:07.209538   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56629.9us
I20260709 13:13:07.172160  2373 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.217327   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:07.213025   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45883.8us
I20260709 13:13:07.204627  2399 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:07.202370  2498 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:07.202366  2486 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:07.234892  2424 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.236642  2432 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.236752   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.237318  2052 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:13:07.238176  2457 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:13:07.238829   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.239787   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.240904  2389 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:07.241566   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.242656   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.242774  2422 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:07.206351  2345 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:07.245008  2041 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.249197   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.249519  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:07.250133   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.256703   927 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.258054  2373 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.259114   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 115) took 3008 ms. Trace:
I20260709 13:13:07.259657   929 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:07.260470  1601 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:07.248204  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:07.281631   924 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:07.270660  1599 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:07.285611   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46885.3us
I20260709 13:13:07.286654   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.251432  2411 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.293066   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.296792   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.296732  2411 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.299186   999 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:07.299710  2411 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.250283  2399 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:07.302788   999 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:07.288981  2375 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:07.307243   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:07.316912   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.264673   926 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.253072  2486 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:07.326570   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57045us
I20260709 13:13:07.286485  2327 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:07.318264   922 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.264529  2432 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.305433   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.342356   999 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:07.264863  2498 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.289750  2389 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:13:07.320725  2380 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.334456  2495 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.325443  2502 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.323185  2411 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.381713   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.356264  2327 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:07.393025   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 106449us
I20260709 13:13:07.394209   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.395233  1564 leader_election.cc:400] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3.
W20260709 13:13:07.395936  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 101246us
I20260709 13:13:07.389703  2459 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.384719  2504 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:07.399859  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:07.400465  1595 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.403776  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:07.385499  2345 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.404609  2345 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.405939   819 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.406384   998 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:07.412922  2500 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:13:07.416800  2498 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.421283   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.422081   923 raft_consensus.cc:2393] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.423048   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:07.423755  2508 consensus_queue.cc:1175] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 1 last_received_current_leader { term: 2 index: 2 } }
I20260709 13:13:07.424861   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:07.425424   930 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.426659  1601 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:07.423952   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.428635   921 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.429061   924 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.428637   927 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:07.424890  2373 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:07.432315  2506 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:13:07.434971  1595 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:07.435743   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:07.435808   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.437727   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:07.438006   928 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:07.436887  2375 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.351294  2325 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.423588   923 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:07.441041   927 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:07.441416   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 222142us
W20260709 13:13:07.447351   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53193.2us
I20260709 13:13:07.447707   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:07.448076   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 184124us
I20260709 13:13:07.453357   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:07.406561  2052 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.460988  2410 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:13:07.493155  2501 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.256s	user 0.003s	sys 0.000s
I20260709 13:13:07.498382  2411 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.499033  2411 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.501139  2411 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:07.506357   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70693.4us
I20260709 13:13:07.507004  2512 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.507366   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:07.507575  2512 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.508085  2512 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.508415  2486 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.508996  2498 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.509605  2498 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:07.270311   749 rpcz_store.cc:276] 0709 13:13:04.250634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.250754 (+   120us) service_pool.cc:224] Handling call
0709 13:13:07.255190 (+3004436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:05.604639 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:05.692038 (+ 87399us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:05.692088 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:13:05.692116 (+    28us) tablet.cc:662] Decoding operations
0709 13:13:05.692693 (+   577us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:05.692747 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:05.692773 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:13:05.693087 (+   314us) tablet.cc:701] Row locks acquired
0709 13:13:05.693107 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:13:05.693260 (+   153us) write_op.cc:270] Start()
0709 13:13:05.693378 (+   118us) write_op.cc:276] Timestamp: P: 1783602785693218 usec, L: 0
0709 13:13:05.693405 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:13:05.696163 (+  2758us) log.cc:844] Serialized 1293 byte log entry
0709 13:13:06.460779 (+764616us) op_driver.cc:464] REPLICATION: finished
0709 13:13:06.775801 (+315022us) write_op.cc:301] APPLY: starting
0709 13:13:06.775954 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:06.776175 (+   221us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:06.783105 (+  6930us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:06.783738 (+   633us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:06.784029 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:13:06.784091 (+    62us) write_op.cc:312] APPLY: finished
0709 13:13:06.784558 (+   467us) log.cc:844] Serialized 35 byte log entry
0709 13:13:06.784788 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.784965 (+   177us) write_op.cc:454] Released schema lock
0709 13:13:06.785027 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:06.785595 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.785622 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":241657,"apply.run_cpu_time_us":3415,"apply.run_wall_time_us":9816,"num_ops":3,"prepare.queue_time_us":387607,"prepare.run_cpu_time_us":70076,"prepare.run_wall_time_us":533253,"raft.queue_time_us":253021,"raft.run_cpu_time_us":56900,"raft.run_wall_time_us":302625,"replication_time_us":718586,"thread_start_us":1389955,"threads_started":4,"wal-append.queue_time_us":448344}]]}
I20260709 13:13:07.514266   930 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:07.514911   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.527747  2380 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:13:07.528630  2424 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:07.529644  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.530337  1003 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.530659  2432 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.532014   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.532801   997 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.536021   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.536758   995 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:07.537284  2410 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:07.537700  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.537823  2410 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.538335  1001 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:07.540537  2420 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:07.540859   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93236.7us
I20260709 13:13:07.541481  2514 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:07.543888   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95300.7us
I20260709 13:13:07.544334   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.540958  2420 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:07.546865   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 98173.1us
I20260709 13:13:07.549106   999 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:07.448881  2342 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:07.438841  2327 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.459254  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:07.550577   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.558015   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.559309   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:07.568150   851 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.558776   849 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:07.454733  2325 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.487128  2306 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.526026  2395 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:07.573637   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.575037   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:07.577720   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 143299us
I20260709 13:13:07.620865   853 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.583778   854 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:07.541482  2410 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:07.624818   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76881.7us
I20260709 13:13:07.461640  2362 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:13:07.555142  2409 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.553886  2420 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.449262  2502 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.553292  2504 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.555008  2461 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:13:07.568274  2052 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.639339  2409 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.639024  2495 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.629700   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:07.665475   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 115647us
I20260709 13:13:07.618206  2342 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.624015  1592 raft_consensus.cc:2376] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3. Re-sending same reply.
I20260709 13:13:07.677508   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:07.672968   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.651680  2504 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.684322   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.690642   853 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:07.666378  2409 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.733850  2504 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:07.734468   995 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.649224  2512 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.663552  2410 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.736271  2325 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:07.702806   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.691057  2498 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:07.737234  2512 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.737449  2306 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.737689  2498 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.739657  2515 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:07.707979  2480 consensus_meta.cc:287] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 13:13:07.740322  2515 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.741062   751 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 3 to 4, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 1 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:07.736001  2410 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.743443  2511 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.705626   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:07.731092  2041 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:07.742697   751 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 2 to 3, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:07.724372   848 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:07.750507   997 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:07.743389   851 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:07.739836  2498 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.760596  2052 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.762164  2327 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:07.750015   751 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:07.769603  1003 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:07.773244   995 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:07.696628  2432 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 13:13:07.789372   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 116484us
I20260709 13:13:07.790843   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:07.765079  2482 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:07.775194   751 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:07.723497  2395 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.762663  2052 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.742648  2515 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:07.805074  2260 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:07.785082   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 234584us
W20260709 13:13:07.808705  2517 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:13:07.772742  2504 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.812774   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.831054   821 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.832347  2513 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.832849  2513 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.833266  2513 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.820429  2519 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.802948   751 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:07.744063  2511 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.828512   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.813881  2052 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.841188   997 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:07.842100   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.846751   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.870641   969 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.872085   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.873126   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:07.787899  2409 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:07.842991   751 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 2 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:07.827746  2260 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.885301   995 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:07.889251   997 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:07.891832   751 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:07.820993  2395 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.836202  2519 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.905849  1004 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:07.840833  2461 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:07.899955   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.900326   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:07.880615  2429 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:07.904682   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 451358us
I20260709 13:13:07.881166  2052 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.885277  2527 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.892022  2508 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.898695   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.913437  1564 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.885634  2389 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.895977  2375 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.894032  2260 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:07.882620   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 257064us
I20260709 13:13:07.895792  2414 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:07.923295   998 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:07.916772   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.887002  2495 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:07.894443  2486 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:07.841120  2504 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.879686  2515 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:07.880082  2335 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:07.929960   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:07.908427  2519 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.914171  2461 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:07.907624  2362 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:07.926393  2389 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.945405   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:07.943689   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45052.5us
I20260709 13:13:07.964116   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:07.970211   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 292763us
I20260709 13:13:07.982188   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:07.982316   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.931130  2411 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:07.942979  2414 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:07.939775  2375 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:07.959988  2461 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.960700  2527 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:08.000149  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70191.3us
I20260709 13:13:07.947867  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:08.009378  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 96051.9us
I20260709 13:13:07.965345  2389 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.015502   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51461.6us
I20260709 13:13:07.998082   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:07.981554  2534 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.021394  2504 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.021320  2534 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:07.996562  2342 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.028795   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:07.949002  2486 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.014251   894 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.035961  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.045120  1601 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:08.038746   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.046729   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:07.980659  2525 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:08.058578   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.063804   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:08.068188   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70193.4us
I20260709 13:13:08.068650   850 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:08.068928   896 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:08.026820   847 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:08.073901  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.031031  2335 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:08.073441  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.083204   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.083354  2528 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.083441   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.084004  2541 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:08.084156   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.084652  2529 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:08.084960  2486 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.030612   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.085212  2525 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:08.085675  2543 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:08.085943  2524 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:13:08.086131  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.086658  2546 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:08.087659  2539 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.089252   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:07.990785  2362 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:08.104068   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 54797.6us
I20260709 13:13:08.147111   998 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:08.111912  2530 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.111763  2375 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.113113  2411 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:08.114760  2534 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.115478   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:08.119099   853 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:13:08.119742  2536 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.122808  2511 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.090277  2495 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.077420  2480 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:13:07.997423  2420 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:08.169489  2521 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.288s	user 0.000s	sys 0.002s
I20260709 13:13:08.169697   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.169966  2531 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.112057  2520 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.170580  2531 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.172722  2531 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.173192  2536 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.173890  2541 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.174383  2541 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.176482  2541 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.158638  2543 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:08.177095   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 177771us
I20260709 13:13:08.177708  2543 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:08.180900  2516 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.526s	user 0.005s	sys 0.000s
I20260709 13:13:08.083238   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.181238   921 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.090814  2335 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:08.185688   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:08.156469   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.149564  2409 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.157233   852 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.157599  1001 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:08.129343  2380 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.129411  2525 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.160159   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:08.124596  1600 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:08.164328  2510 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.169320  2540 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:08.127826  2306 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:08.145853   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.197167   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.157220   853 pending_rounds.cc:85] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:13:08.191695  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.198716   853 pending_rounds.cc:107] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:13:08.199150   853 raft_consensus.cc:2889] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:08.191696   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.192562   921 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.159418  2546 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:08.203464   929 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.192788   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.211643   997 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:08.195525   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.189198   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.181212   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.175529  2536 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.148151  2529 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.146436  2414 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.129240  2528 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.194047  2411 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:08.153592  2539 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.213294   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:08.214722   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.215267   851 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:08.196758  2525 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.220449  2554 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:08.289752  2432 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:13:08.300225  2554 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.289811  2325 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.301723  2325 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:08.302572  2325 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.218407  2514 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:08.305816  2420 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.305812  2514 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:08.306470  2420 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:08.189594  2515 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.193933   848 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:08.277604   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.290302   853 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:08.311672   921 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:08.217494  2362 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.218780  2380 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.219473   852 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:08.219882   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.319937   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:08.241725   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81626us
I20260709 13:13:08.221107  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:08.231786   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.303442  2511 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:08.304445   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:08.304560  2409 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:08.218700  2260 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.351359   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:08.352105  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131068us
I20260709 13:13:08.352294  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.352878  1002 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:08.322639   850 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:08.349879   855 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:08.355343   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.359233  2533 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:08.360005   926 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:08.360184   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:08.360342  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:08.360935  1601 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.361160   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.220754  2512 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:08.220029  2410 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.318380   922 raft_consensus.cc:1813] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Handling vote request from an unknown peer 3bb5726887244caca146e176cc6bd070
I20260709 13:13:08.231140  2519 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.366080   846 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:08.366142   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.201742  2540 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:08.355775  2424 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:08.219396  2052 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.309500  2325 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.329092  2536 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.307367  2555 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:08.369639   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.370580   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:08.371685   921 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:08.373447   854 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:08.309852  2515 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.302590  2554 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.228045  2510 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.363044  2362 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:08.228574  2513 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.395480   923 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:08.396164   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:08.396435   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.397013   923 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:08.397341  2513 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.398113   923 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:08.275655  2306 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.379594  2424 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.374781   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.375444  2540 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.365656  2512 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:08.309216  2539 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.385859   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.406162   849 raft_consensus.cc:2368] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:08.407599  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:08.399731  1601 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:08.220338  2335 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.408841   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.391361  2520 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.386467  2555 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:08.393265  2531 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:08.417117   819 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:08.420405   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.399627  2298 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.411072   922 raft_consensus.cc:2410] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:08.435410  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:08.440289   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.405710   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.440583  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:08.440256   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 112385us
I20260709 13:13:08.449751  1599 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:08.412969  2424 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.375588  2527 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:13:08.415370  2395 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.418596  2484 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:08.233031  2546 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.496739   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.497073   926 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:08.497514  2410 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:08.495854  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 60552.6us
I20260709 13:13:08.420900  2389 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.444271   847 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:08.485575  2052 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.492691  2515 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.482681   928 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.496071   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:08.487540   930 raft_consensus.cc:2410] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:13:08.449967   967 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.535s (SENT)
I20260709 13:13:08.484494  2529 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.486655  2512 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.482116  2547 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.636116   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.482582  2548 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:13:08.441617  2531 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:08.443265  2298 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:08.692137  2531 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.692782  2298 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.693898  2298 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.670428  2325 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.411742  2536 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.695830  2375 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.695953  2325 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.696426  2375 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.697336   922 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:08.698235  2536 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.452107  2411 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.698987  2515 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.645864   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.700063  2504 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.700567  2504 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:08.500118  2564 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:13:08.637241  2484 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.456952  2549 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.228s
I20260709 13:13:08.701624  2484 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.702474  2504 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.638015  2395 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.672642   895 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.451382  2331 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:08.676093   929 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:08.678423   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:08.671753  2461 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 13:13:08.698480  2375 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:08.675323   995 raft_consensus.cc:2368] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:08.706861  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.710709   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.652231  2409 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.711527  1004 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:08.719545  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.702265  2525 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.702968  2380 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.707494  2399 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.707748  2327 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.726709  1599 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:08.703014  2504 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.726027   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.737509   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:08.699515  2515 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.720469  2345 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:08.677371  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:08.720671  2260 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.756371   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.722666  2520 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:08.760293   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:08.713003  2395 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.714250  2461 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.712110  2554 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:08.717778  2409 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.728256  2414 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.721848  2375 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.736407  2527 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:08.735180  2331 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:08.757136  2327 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.758152  2310 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:08.680720  2555 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.765226  2519 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.773283  2573 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.774115  2560 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:13:08.762285  2504 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.759528  2345 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.740587  2399 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.776393  2538 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:08.740576  2530 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:08.776892  2495 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.774096  2527 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.777938  2525 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.778767  2424 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.703569  2547 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.781440  2375 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.768587  2549 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:08.740628  2380 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.881906   855 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:13:08.903048   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 138986us
I20260709 13:13:08.900740  2555 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.809270  2395 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:08.800897  2429 consensus_meta.cc:287] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.871s	user 0.000s	sys 0.010s
I20260709 13:13:08.765705  2520 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.902060  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.788841  2510 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:08.810695  2553 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.648s	user 0.004s	sys 0.000s
I20260709 13:13:08.792770  2461 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.793366  2335 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.901722  2424 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.881183   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.796766  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.876212  2310 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.872155  2422 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.793004  2536 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.780478  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 109165us
I20260709 13:13:08.800504  2502 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:08.906524   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 456555us
I20260709 13:13:08.782069  2554 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.929644  2515 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.929839   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1339) took 1092 ms. Trace:
W20260709 13:13:08.923271  2552 consensus_queue.cc:1175] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 4 status { last_received { term: 1 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:08.930217   999 rpcz_store.cc:276] 0709 13:13:07.837117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.837272 (+   155us) service_pool.cc:224] Handling call
0709 13:13:07.837594 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.904385 (+ 66791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.904414 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:07.922766 (+ 18352us) log.cc:844] Serialized 26 byte log entry
0709 13:13:08.062067 (+139301us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:08.062470 (+   403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.062541 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:08.809857 (+747316us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:08.929554 (+119697us) raft_consensus.cc:1685] finished
0709 13:13:08.929576 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.929765 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9877,"raft.queue_time_us":20629,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":18631,"spinlock_wait_cycles":2483968,"thread_start_us":105513,"threads_started":1,"wal-append.queue_time_us":226046}
I20260709 13:13:08.932480  2536 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.932919  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:08.934748  2536 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.935415  2551 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.937752  2554 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.938313  2554 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.938704  2512 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.940338  2554 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.940824  2554 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.941233  2554 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.943133  2554 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:08.944060  2540 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.001s
I20260709 13:13:08.944721  2375 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.944492  2540 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.880053  2573 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.916841   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.807427  2533 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:08.947994   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 187806us
I20260709 13:13:08.916115  2525 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.877398  2531 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.791144  2331 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.921507   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.790679  2325 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.898988  2549 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:08.881951  2547 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.914953  2429 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:08.928092   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.962769   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:08.963092  2508 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:08.963577  2508 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.963937   968 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:08.952827  2424 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.964717  2502 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:08.966043   999 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:08.955260  2573 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.919214  2504 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.923283  2520 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.926395  2310 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.958308  2531 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.993875  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.969638   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.995510   999 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:08.969976  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:08.958985   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:08.996064  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:08.996546  1002 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.996732  1601 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:09.000033   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:09.000414  2484 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:08.989617   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.002842   853 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:09.003572  1002 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:09.004050   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:08.801965  2514 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.973280  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.006101  1000 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.974511   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.006582   896 leader_election.cc:400] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:08.964854  2424 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.977380   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:08.982434  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.983031   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:08.977319  2375 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:09.016153  2375 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.952497  2536 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:08.956058   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:09.017033   922 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.016923  2536 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.018115  2389 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.018771  2515 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.019153  2515 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:09.019237  2536 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:08.989357  2345 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.958034  2527 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.958786   847 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:09.022050   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.022419   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.023433   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:08.956875   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:09.024845   925 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:09.025712   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:09.026404   925 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:09.026590   998 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:08.989701  2508 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:08.952539  2533 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.959482  2549 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:08.991395  2570 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 13:13:08.991514   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:08.991873  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.952091  2422 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:08.992457   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:08.965139  2502 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.006891   896 leader_election.cc:403] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:09.006975   996 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.946718  2540 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.008787  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:08.987701  2399 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:08.971405  2525 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.016607  2375 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.028156   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.064936   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:09.065853   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.024075   847 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:09.047945   997 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
W20260709 13:13:09.068763   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 62311.5us
I20260709 13:13:09.069640   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.054634  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:09.048228  1600 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:09.047097   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.066370   853 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:09.054174   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:09.019549  2515 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:09.063756   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 71383.2us
I20260709 13:13:09.076928   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.071028   925 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.072103  2536 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.082230   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.082814   853 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.058696  2502 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.074267  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:09.081079  1000 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:09.053704  2524 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:09.072701  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:09.096330   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.073446  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:09.086598   922 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:09.087029  2508 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:09.087349  2558 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.459s	user 0.000s	sys 0.001s
I20260709 13:13:09.016539  2538 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:09.088585  2525 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.088791 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:09.112891  2530 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:09.113176  2484 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.113461  2530 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.113755  2310 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.089926  2375 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.091337   997 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:09.124262   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.124919   851 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:09.094045   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.102639  1002 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:09.092226  2529 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:09.120980   925 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:09.115720   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 52817.9us
I20260709 13:13:09.130388   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.131453   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.131665  2510 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:09.132273  2510 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:09.133989   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:09.134459   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.134022   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.127715  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.139145   853 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:09.140779   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.137441   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.137190   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.143527   997 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.144963  1001 raft_consensus.cc:2368] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4.
I20260709 13:13:09.145812   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.146312   929 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:09.147238   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.147951   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.148207   853 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:09.139675  2052 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.147037   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.151701   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.154768   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.129655  2529 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.147078  2432 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.160504  2052 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.161545   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.170432   851 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:09.167124   997 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:09.139937  2504 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:09.168972  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1270) took 1072 ms. Trace:
I20260709 13:13:09.135807  2579 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.171761   895 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.172685  2052 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:09.176673   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.143893  2422 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.171754  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:09.177178   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.146871  2342 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:09.184912   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.173183  1004 rpcz_store.cc:276] 0709 13:13:08.096584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.096905 (+   321us) service_pool.cc:224] Handling call
0709 13:13:08.104760 (+  7855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.714305 (+609545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.168746 (+454441us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540035aa54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:09.168917 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1394753408}
I20260709 13:13:09.148653  2395 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:13:09.179800  2577 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.170032  2432 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.184028   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.143374  2512 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.187433   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.168924  2345 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:09.191614   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.172358  2529 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.195307   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.192739  2410 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:09.190546  2432 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.143234  2389 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:09.175510  2540 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.201164   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.178182  2525 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.148780  2573 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:09.177537  2414 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.173589   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:09.204097  2546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.007s
I20260709 13:13:09.211685   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.196981  2504 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.135450  2510 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.243198   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.220679   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.244084  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:09.200462  2424 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.238801  2310 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.249025  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:09.233043  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 182670us
I20260709 13:13:09.255254  1003 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:09.239835  2306 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:09.247491   849 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1276) took 1196 ms (client timeout 1526 ms). Trace:
I20260709 13:13:09.248410   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.258091   997 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:09.257841   996 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:09.248694  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.259814  1004 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:09.248762   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:09.224396   923 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:09.257156   849 rpcz_store.cc:269] 0709 13:13:08.050952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.063588 (+ 12636us) service_pool.cc:224] Handling call
0709 13:13:09.247226 (+1183638us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400177ad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:09.247398 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707805440}
I20260709 13:13:09.265949   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.266625   851 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:09.267654   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 80334us
I20260709 13:13:09.268474   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.269541   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:09.214243  2546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.007s
I20260709 13:13:09.272934   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:09.275692  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.276348  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.292999  1002 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.275861   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.293637   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.274290   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.294914   999 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.209887  2577 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:09.300751  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.230594  2528 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.212777  2540 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:09.302914  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:09.309682  1601 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.302011  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.308071   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.229432   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.230070  2530 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.313200   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.315475   846 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.313068   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.246099  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:09.313901   895 leader_election.cc:400] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:09.273444  2375 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.273193  2414 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.317476   853 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:09.247285   847 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:09.314217  1004 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:09.274411  2389 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.319238  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.317664   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.299301  2573 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.217928  2052 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.332484   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.322063  1002 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:09.329950   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.336740   997 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:09.337123  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.311550  2424 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.273910  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.333509   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.342510   846 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:09.340909   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.345750   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:09.338464  1601 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:09.338025  1002 raft_consensus.cc:2368] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:09.347875   968 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:09.349419  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75596.6us
I20260709 13:13:09.350085  2502 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.352905  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:09.353603  2585 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:13:09.354003  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.355110   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.355253  2577 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.355999   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.356247  2550 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:09.356701  2550 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:09.357039  2550 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:09.359750  2538 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.362671   997 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:09.358857   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.347955  2410 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:09.365396  1000 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:09.366459  2512 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.366052  2410 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.367022  2512 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:09.319278  2345 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:09.368044  2512 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.369103  2052 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.337795  2504 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.370210  2504 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.371786   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:09.372287  2540 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.372764  2538 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:09.373255  2538 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.373626  2538 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:09.379667   999 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:09.350235  1566 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:09.339293  2306 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.339172  2530 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.347538  2511 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:09.385246  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:09.396147   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 98884.8us
I20260709 13:13:09.385493   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.399441   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:09.400241   846 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.381167   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.402088   896 leader_election.cc:400] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:09.402359   896 leader_election.cc:403] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:09.398366   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.406275   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.416396   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.417932   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.383038  2389 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:09.383119  2573 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:09.344100  2549 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.439738   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.441221   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.442027   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.442497   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.419476  2052 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.388841  2306 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:09.385891  2528 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:13:09.437666  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.403959  2511 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.446765   929 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:13:09.453118   929 pending_rounds.cc:85] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 1
I20260709 13:13:09.453380   929 pending_rounds.cc:107] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:13:09.453670   929 raft_consensus.cc:2889] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:09.421718  2375 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.403398  2530 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:09.421885  2543 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:09.445487  1002 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:09.413462  2414 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:09.483745  1597 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:09.483948  2306 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.484937  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:09.485518  1599 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.457748   846 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:09.490200  2432 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.454775  2511 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.472473   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.446617  2547 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:13:09.444823  2389 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.455334   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.456367  2534 raft_consensus.cc:2764] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:09.444327   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:09.458411   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.493402  2549 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.464377  2502 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.464512  1595 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:09.482987  2529 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.459224  2594 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:09.503466   923 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:09.577585  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:09.505836   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 99647.8us
I20260709 13:13:09.578692  1004 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:09.578948   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.580608   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.581555  2577 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.507187  2576 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.509131  2530 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.510757   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.584748   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:09.478672  2411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.009s	sys 0.000s
I20260709 13:13:09.515532  1599 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:09.460026  2593 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 13:13:09.586766  2411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.010s	sys 0.000s
I20260709 13:13:09.591980   929 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:09.512713  2414 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.607915   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.524237  2536 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
W20260709 13:13:09.595537  2515 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.002s
I20260709 13:13:09.614571  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:09.524214  2429 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.591867   847 raft_consensus.cc:606] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:09.576506  2573 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:09.593515  2508 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:09.573741  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 223569us
I20260709 13:13:09.611686  2548 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:09.625198  2525 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.639868   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:09.630559  2515 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.598580  2550 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.635936  1592 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:09.658334   998 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:09.658454  2515 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.660760  2529 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.661345  2536 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.624603  2577 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:13:09.662395  2573 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.633370  2052 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:09.682583  2052 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.611635  2414 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.597535  2594 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.676972   926 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:09.680568  2432 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.680653  2536 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:09.690335  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:09.702860  1004 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.704335  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:09.686538   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:09.632447  2429 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.642163  2512 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.711159   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:09.683897  2568 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.997s	user 0.001s	sys 0.004s
I20260709 13:13:09.638427  2508 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.719985   854 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:09.717063  2515 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:09.727398  2515 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:09.728453  2515 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.729959   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:09.731252   854 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:09.684856  2052 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.732496   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:09.642491  2548 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:09.754185  2536 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.756210  2409 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.756970  2580 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.757488  2580 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.757622  2548 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:09.757961  2580 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.765671  2424 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:09.773470  2414 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.774364  2550 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.775255  2429 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.777266  2508 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:09.784410   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.787497  1004 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:09.683504  2538 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.803651   922 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:09.805874  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:09.763268  2533 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.000s
I20260709 13:13:09.737529  2502 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:09.801983  2409 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:09.724890  2512 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:09.820173   927 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:09.819741  2549 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:09.818485  2502 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:09.820674  2533 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:09.954905  2560 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.746s	user 0.002s	sys 0.000s
I20260709 13:13:09.959769  2594 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.956611   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.962335   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.934376  2502 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:09.964726   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.965837   851 raft_consensus.cc:2410] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:09.939133  2549 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:09.971068   852 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.971578   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:09.970697  2538 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.981843   849 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:09.984882  2512 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:09.988066   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.989682   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.990604  2594 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.990931   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:09.991256  2594 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.991922  2594 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:09.995615  2389 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:09.998411   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.000028   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.005539   851 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:10.007831   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:10.010035  2515 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.011039  2515 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:10.016165   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.019567   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.017555  2594 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:13:10.022028   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:10.023437   998 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:10.026041   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.009119  2409 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.009622  2298 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.032949  2573 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:10.025840  2345 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.038203  2533 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.025606  2508 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:10.025244  2504 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.048087   926 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:10.019346  2389 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:13:10.057009   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:10.050882  2562 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.602s	user 0.005s	sys 0.014s
W20260709 13:13:10.051653  2511 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.007s
I20260709 13:13:10.073664   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:09.997267  2549 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.054436   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:10.081190   923 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:10.075523   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.041163  2580 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.041450  2298 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.096175   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.097476   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.098093   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:10.098909   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.102162   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.107836   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.082021  2577 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:10.091074  2594 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.111106   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.080967  2375 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:13:10.080250  2511 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.120312   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:10.124147   928 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:10.109093  2504 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.128701   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.129776  2508 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:10.130326  2508 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:10.131309  2508 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.132740  2508 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:10.121315  2577 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.138269   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.138792   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.140769   846 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:10.134111  2573 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:10.127745  2345 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:10.137751  2594 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:10.141929  2414 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:10.144501   847 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:10.144793  2504 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:10.146032  2538 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:10.156248   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.147928  2594 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.146057  2395 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:13:10.157739  2594 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:10.166033  2511 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.167455  1594 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:10.168515   922 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:10.168951   922 pending_rounds.cc:85] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 1
I20260709 13:13:10.169186   922 pending_rounds.cc:107] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:10.169423   922 raft_consensus.cc:2889] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.170547  2502 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.171854  2594 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.172708  2502 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.173885  2594 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.163214  2298 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.162340   846 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:10.174587   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.174960   995 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:10.175132   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.175499  1596 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:10.176504   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:10.176663   851 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:10.178673  2594 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:10.179214  2594 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:10.179080  2573 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:10.180161  2594 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.188246   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1306) took 1042 ms. Trace:
I20260709 13:13:10.188737   850 rpcz_store.cc:276] 0709 13:13:09.146089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.672343 (+526254us) service_pool.cc:224] Handling call
0709 13:13:09.680080 (+  7737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.754754 (+ 74674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.754786 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:09.754826 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:09.754855 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.187982 (+433127us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b54003f02d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:10.188030 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:10.188180 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171285760}
I20260709 13:13:10.192179   853 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:10.192588   853 pending_rounds.cc:85] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 1
I20260709 13:13:10.192806   853 pending_rounds.cc:107] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:10.193019   853 raft_consensus.cc:2889] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.195132   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1297) took 1215 ms. Trace:
I20260709 13:13:10.200249   853 rpcz_store.cc:276] 0709 13:13:08.980002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.191433 (+1211431us) service_pool.cc:224] Handling call
0709 13:13:10.191807 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.193427 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.195089 (+  1662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:10.213208   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:10.214119   929 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:10.219962   819 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.225383  2508 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:10.227730  2533 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.228358  2533 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.230576  2533 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.232374   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.253101   929 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:10.255038   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:10.158522  2538 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.242511  2508 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.248351  2530 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.253773  2548 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.256875  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:10.264850  2395 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:10.278522   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:10.280125  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.281883  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.283296  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.283874   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.288551   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:10.290647  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.293756   998 raft_consensus.cc:2368] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3.
I20260709 13:13:10.294811   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:10.291757  1004 raft_consensus.cc:2410] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:10.308223  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:10.308054   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:10.309338   999 raft_consensus.cc:2410] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:10.318387   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:10.292059  2567 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.268918  2508 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.293102  2395 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:10.327466   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49034.2us
I20260709 13:13:10.318310  2577 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:10.322750   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 89293.8us
W20260709 13:13:10.339010   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 59684.5us
I20260709 13:13:10.339857   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:10.342681  2395 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.365393  2527 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:10.366165  2527 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.368641  2536 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.369246  2536 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.328059  2567 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.389770  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:10.392405  2640 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.393167  2640 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.394449  2548 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:10.395078  2548 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:10.396456   852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1295) took 1442 ms (client timeout 1539 ms). Trace:
W20260709 13:13:10.397022   852 rpcz_store.cc:269] 0709 13:13:08.953733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.958760 (+  5027us) service_pool.cc:224] Handling call
0709 13:13:10.396256 (+1437496us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54000c99d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:10.396374 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2240905600}
W20260709 13:13:10.402333  2634 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:10.404364  2548 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.410374  2508 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.416810  2567 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:10.419301   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 86559.2us
I20260709 13:13:10.353452  2638 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:10.421487   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93348.4us
I20260709 13:13:10.373641  2635 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.435745   998 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:10.426523  2638 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:10.397120  2536 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.440279  2638 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:10.447175   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1261) took 1015 ms. Trace:
I20260709 13:13:10.419665  2538 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.440759  2641 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:10.447657   927 rpcz_store.cc:276] 0709 13:13:09.438863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.442203 (+  3340us) service_pool.cc:224] Handling call
0709 13:13:10.446972 (+1004769us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400082854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:10.447110 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866637568}
I20260709 13:13:10.455924   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.456740   922 raft_consensus.cc:1813] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Handling vote request from an unknown peer 3bb5726887244caca146e176cc6bd070
I20260709 13:13:10.456789  2527 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.457540   922 raft_consensus.cc:2410] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:10.429436  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:10.458611  2593 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.497572   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:10.457858  2640 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:10.533249   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 198049us
I20260709 13:13:10.535220   929 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:10.535902   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.536366   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.542750   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.545867   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:10.548990   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.549405   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1319) took 1366 ms. Trace:
I20260709 13:13:10.550800   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:10.550990   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:10.553998   927 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:10.553515   969 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.496824  2594 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.549831   854 rpcz_store.cc:276] 0709 13:13:09.183141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.725417 (+542276us) service_pool.cc:224] Handling call
0709 13:13:09.727661 (+  2244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.731819 (+  4158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.549164 (+817345us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b54000726d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:10.549345 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7533440}
I20260709 13:13:10.610926   928 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:10.441746  2635 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.496608  2567 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:13:10.611426   895 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.539s (SENT) [suppressed 4 similar messages]
I20260709 13:13:10.543807  2573 raft_consensus.cc:2764] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:13:10.621939   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 67028.6us
I20260709 13:13:10.496438  2593 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.671708  2549 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.672323  2549 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.674039   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.672609  2593 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.674655   930 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
W20260709 13:13:10.681329   930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1266) took 1240 ms (client timeout 1574 ms). Trace:
I20260709 13:13:10.673768   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.633440   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.629030   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:10.679037   849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:10.682528   851 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:10.682814   849 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
W20260709 13:13:10.681671   930 rpcz_store.cc:269] 0709 13:13:09.440421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.447714 (+  7293us) service_pool.cc:224] Handling call
0709 13:13:10.670902 (+1223188us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b5800010598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:10.681275 (+ 10373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3436800}
W20260709 13:13:10.670197  2639 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:13:10.669545  2646 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:13:10.625643  2648 raft_consensus.cc:993] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:10.622057  2649 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.741879  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.743069  1003 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:13:10.706307   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 275845us
I20260709 13:13:10.626459  2345 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:10.621691  2538 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:10.706940   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95531.6us
I20260709 13:13:10.705976   851 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:10.738157   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.753927  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:10.753715   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:10.764079   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.629019  2520 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:13:10.776804   998 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:10.778003   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:10.765630  2604 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.061s	user 0.001s	sys 0.001s
W20260709 13:13:10.783209   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 403738us
I20260709 13:13:10.752615  2567 raft_consensus.cc:1081] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:10.785410   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:10.704012  2594 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.774503  2546 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:10.766928  2593 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:10.834731  2549 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.835419  2549 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:10.819142  2548 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:10.836411  2548 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:10.837661  2515 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.835992  2549 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.776252  2538 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.779174  2649 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.841691   853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:10.843459   853 raft_consensus.cc:606] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:10.823948  2655 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:13:10.820679  2536 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:10.808018  2593 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.851886  2635 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.852219  2593 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.855748  2548 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.862421  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:10.864879  2549 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.865641  2653 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:10.865700   922 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:10.866288  2653 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.867328   922 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:10.869866   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:10.868427  2414 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.871567  2414 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:10.873592  2414 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:10.874253  2649 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:10.817955  2658 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:10.843520  2373 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.877678  2345 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:10.838415  2515 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.888268  2373 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:10.899510  2594 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.868461  2653 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:10.913398   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 149413us
I20260709 13:13:10.908013  2520 consensus_queue.cc:1243] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:10.884724  2345 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:10.904829   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:10.905253   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 115724us
I20260709 13:13:10.933179   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:10.934623  2658 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:10.938733  2668 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:10.926561  2345 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:10.908474  2399 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:13:10.961259   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:10.953600  2567 raft_consensus.cc:2764] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:13:10.912533  2414 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:13:10.956084  2536 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:10.969383  2515 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:10.974601   821 leader_election.cc:400] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4.
I20260709 13:13:10.955075  2546 raft_consensus.cc:697] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:10.978312   821 leader_election.cc:403] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:10.966877  2667 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.996814   821 leader_election.cc:400] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3.
I20260709 13:13:10.997299   821 leader_election.cc:403] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:10.993302  2536 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:10.967236  2662 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:11.001634  2536 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.006306   854 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:11.007270   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.007784   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.002609   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:10.939661  2668 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:11.000348  2638 consensus_meta.cc:287] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.553s	user 0.004s	sys 0.007s
I20260709 13:13:11.029335   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.031225   851 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:11.038965   846 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:11.039199   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.000350  2567 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3.
I20260709 13:13:11.041127   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.015645   930 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:11.018448  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1370) took 1444 ms. Trace:
I20260709 13:13:11.021632   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.044761   852 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:11.022500   850 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.040957  2573 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:13:10.998494  2666 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4.
I20260709 13:13:11.012804   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.044447  1002 rpcz_store.cc:276] 0709 13:13:09.574268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.576836 (+  2568us) service_pool.cc:224] Handling call
0709 13:13:09.577218 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.592870 (+ 15652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.592913 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:09.592966 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:09.593019 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.018205 (+1425186us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54003e4754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:11.018251 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.018389 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35438976}
I20260709 13:13:11.064797  2670 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.067106  2670 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.069669  2670 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.086515   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.088110   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.090655  1004 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:11.093525   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.003959  2667 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:11.114835  2682 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:11.115444   853 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.116711  2667 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.105350  2515 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.117703  2515 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.120018   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.109710   854 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.109846   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.122989   996 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:11.123994   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.125025   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.125376  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 215) took 3004 ms. Trace:
I20260709 13:13:11.126013  2373 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.126211   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.065510   922 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:11.126820  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 365) took 2177 ms. Trace:
I20260709 13:13:11.066282  2345 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.065250  2546 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.113292   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.041118  2638 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.113888   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:11.107043  2647 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:13:11.125200  2668 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:11.112474   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.127250   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.129966   855 raft_consensus.cc:2376] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3. Re-sending same reply.
I20260709 13:13:11.144197   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.140130   853 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
W20260709 13:13:11.145654   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 203395us
I20260709 13:13:11.127358  1595 rpcz_store.cc:276] 0709 13:13:08.949387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.952699 (+  3312us) service_pool.cc:224] Handling call
0709 13:13:08.964264 (+ 11565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.573582 (+609318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.126523 (+1552941us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54006f22d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:11.126750 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126099200}
I20260709 13:13:11.146670  2534 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:11.145489  2592 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:11.149035  2592 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.149842  1595 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:11.151461  2667 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:11.154309   929 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:11.125963  1596 rpcz_store.cc:276] 0709 13:13:08.120630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.145862 (+ 25232us) service_pool.cc:224] Handling call
0709 13:13:08.161369 (+ 15507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.175972 (+2014603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.125081 (+949109us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400270954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:11.125271 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4631203584}
I20260709 13:13:11.152961   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:11.129925  2373 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.160830   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:11.151454  2592 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.158419  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:11.155337  2515 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.162213  2668 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:11.156442  2673 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:13:11.193002   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:11.172750   997 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:11.196473   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:13:11.197028   997 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:11.197374   997 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:11.198653  2638 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.157349  2669 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:13:11.167189  2373 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.200008  2669 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:11.200477  2669 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:11.201481  2373 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.175240   998 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.201702  2669 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.204550   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.204733   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:11.205988   923 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:11.175810  2549 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:11.180806   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.188092  2681 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:11.188778  2683 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:11.191293  2680 consensus_queue.cc:1059] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:11.207556   998 raft_consensus.cc:2410] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:11.222538   997 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.211246   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:11.214278   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.213984   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1305) took 2036 ms. Trace:
W20260709 13:13:11.223197   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 313141us
I20260709 13:13:11.230764   996 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:11.247582   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:11.222103  2669 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:11.214704  2549 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.260563   851 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:11.220815  2656 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:11.261601   997 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.215265  2515 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:11.263036   849 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.271138   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.283594   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.221071  2504 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.232892  2373 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:11.313405   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.245616  2643 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:11.285259   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.242204  2683 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.471419   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.361075   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.283141  2641 raft_consensus.cc:3055] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:11.487193   849 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:11.493801   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.503109   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.503764   854 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:11.504065   854 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:11.258702  2511 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.244s
I20260709 13:13:11.486529  2641 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:11.283150  2310 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:11.509570   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:11.480762  2684 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:13:11.516937   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:11.509712  2641 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.523576   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:11.515573   851 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:11.314385  2549 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:11.516877   854 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.255923  2648 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.251s
I20260709 13:13:11.552596   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:11.556746   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 241862us
I20260709 13:13:11.245460   847 rpcz_store.cc:276] 0709 13:13:09.177608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.681307 (+503699us) service_pool.cc:224] Handling call
0709 13:13:09.681673 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:10.154004 (+472331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.213732 (+1059728us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540023a854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:11.213927 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042772864}
I20260709 13:13:11.560396   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.526168  2414 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:13:11.510269  2310 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.525502   851 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.572239  2041 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:11.574115   851 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:11.485572  2504 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.485896  2373 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.485644  2656 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.486148  2515 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.559821  2641 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:11.584671  2648 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:11.588361  2504 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.594503  2549 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.596997  2310 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:11.597988  1001 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.600718  2683 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.602174  2536 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:11.578871   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.620507   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.621445  2649 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:11.621706   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.599450   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.621873  2649 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:11.623773  2685 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:11.630823  2648 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:11.632045  2649 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.633098  2514 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:11.634521   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 112) took 1007 ms. Trace:
I20260709 13:13:11.635102  2515 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:11.635623  2511 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:11.636806  2373 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:11.638455  2530 raft_consensus.cc:993] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
W20260709 13:13:11.639338  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 299339us
I20260709 13:13:11.653910  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:11.654170  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.654419  2683 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:11.671490  1002 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:11.654668  1595 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:11.655094   848 rpcz_store.cc:276] 0709 13:13:10.627344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.666924 (+ 39580us) service_pool.cc:224] Handling call
0709 13:13:11.599231 (+932307us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:11.634452 (+ 35221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9418496}
I20260709 13:13:11.655577  1003 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.666028   851 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:11.669087   999 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:11.669117   922 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:11.679093  1564 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:11.687901   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 183532us
I20260709 13:13:11.623945  2536 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.689890   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:11.693260   930 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:11.698369   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.696732  2504 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:11.706930   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.707051  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:11.678442  2546 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.681537  2649 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:11.711774  2620 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:11.712857  2648 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:11.714625  2696 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:11.695704  2670 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:13:11.717412  2685 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.670378  2373 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.686659  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.764178  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.697594  2310 raft_consensus.cc:1081] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:13:11.709167  2504 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.784291   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.784849   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.785027   853 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:11.785485   851 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:11.800601  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:11.805155   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:11.809500   852 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:11.818496  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:13:11.819367  1002 raft_consensus.cc:606] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.820443  1002 raft_consensus.cc:614] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:11.755766  2546 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.775147  2508 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:13:11.787956   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:11.773847  2373 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.782195  2504 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:11.818078  2640 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.851313  1002 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:11.763808  2685 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.826725  2567 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:11.863629  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 184579us
I20260709 13:13:11.846483  2549 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.847045  2656 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:11.870075  2695 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:13:11.863155  2375 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:13:11.857844  2640 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.885383  2685 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.892553  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.902181  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.898701  2580 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.929790  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.930265  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.931818  2704 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:11.932286  2704 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:11.932698  2704 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.933010  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.933781  2310 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.934016  1592 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:13:11.925354  2673 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:11.934333  2310 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.935913  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.943717   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:11.944267  2310 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:11.926167  2536 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.945036  2310 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.945184  2640 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.945559  2310 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.946938   924 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:11.949419  2375 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.949960  2375 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.924996  2298 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:11.951584   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1418) took 1271 ms. Trace:
I20260709 13:13:11.951702   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:11.951960  2702 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:13:11.952741  2373 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:11.956494  2653 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:11.957150   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:11.957792   925 raft_consensus.cc:2376] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4. Re-sending same reply.
I20260709 13:13:11.958308  2306 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:11.958716  2306 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:11.954119   921 rpcz_store.cc:276] 0709 13:13:10.680568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.680706 (+   138us) service_pool.cc:224] Handling call
0709 13:13:10.680987 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.871806 (+190819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.871835 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:10.871874 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:10.871904 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.951266 (+1079362us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400327dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:11.951314 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.951430 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438378880}
W20260709 13:13:11.960975   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 155997us
I20260709 13:13:11.959385  2577 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.968416  2502 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:13:11.968386  2399 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.972062  2306 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.980973   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:11.982947  2713 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:11.981312  2577 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.984184  2713 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:11.984562  2705 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:11.981598  2373 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:11.985019  2705 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:11.985289  2713 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.985698  2373 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.986155  2705 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.964600  2580 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:11.990518  2580 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:11.991284   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.970652  2546 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:11.987542  2705 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:11.991402  1596 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:11.964506  2298 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:11.992380  2508 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:12.021302  2298 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:11.992612  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:11.987823  2399 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.993625  2306 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.028120  2666 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:12.028276   922 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:12.028650  2666 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:12.029562  2666 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.031095  2666 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:11.994809   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.033844  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.034845  1593 raft_consensus.cc:2393] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:12.035987   968 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.036703  2538 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:12.038712  2399 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:12.039882   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.997741  2504 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:11.998730  2375 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.002036  2704 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:12.003362  2310 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:12.006731  2692 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:13:12.008073  2373 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:12.009446  2650 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 1.243s	user 0.000s	sys 0.003s
I20260709 13:13:12.010351   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1294) took 1145 ms. Trace:
I20260709 13:13:12.064710   929 rpcz_store.cc:276] 0709 13:13:10.864613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.865580 (+   967us) service_pool.cc:224] Handling call
0709 13:13:10.865874 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.156980 (+291106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.010095 (+853115us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54003e1554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:12.010286 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662766336}
I20260709 13:13:12.012931  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.041414  2424 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.066785  2424 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.068068   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.069022  2424 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.069314   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.070530   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.062460  2577 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.073570  2577 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.074079  2649 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:12.074226  2577 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.039354  2399 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.077087  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.078400  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.043833   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:12.077808  1004 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:12.081619   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.082726   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.084328   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:12.084923   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.085713   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.085837   853 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:12.069612  2424 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:13:12.101897  2504 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.099298  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:12.087863   929 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:13:12.098063  1003 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:12.108814   929 pending_rounds.cc:85] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:13:12.109817   929 pending_rounds.cc:107] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:13:12.097597  2546 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.110154   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.111127   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.087312  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.112079   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.112409   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.098331   927 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:12.113559   927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42390
I20260709 13:13:12.090879   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.110177   929 raft_consensus.cc:2889] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:12.105468  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.114087   927 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:12.116118   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1417) took 1483 ms. Trace:
I20260709 13:13:12.100885   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:12.117331   927 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:12.095846  2653 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.118521  2711 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.082610  2414 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.119072  2711 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.119385  2414 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.085413  2538 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:13:12.121423  2711 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.121711  2414 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.116868   928 rpcz_store.cc:276] 0709 13:13:10.632660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.632789 (+   129us) service_pool.cc:224] Handling call
0709 13:13:10.829483 (+196694us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b5800010598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:10.829690 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.853259 (+ 23569us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:10.957987 (+104728us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.141503 (+183516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.141735 (+   232us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:11.141792 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.115896 (+974104us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54001f0754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:12.115945 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.116068 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7365,"prepare.queue_time_us":82799,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":58429,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"spinlock_wait_cycles":56857728,"thread_start_us":251758,"threads_started":3,"wal-append.queue_time_us":63274,"wal-append.run_cpu_time_us":1761,"wal-append.run_wall_time_us":1093939}
I20260709 13:13:12.124974   928 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:12.097704  2704 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:12.127918   926 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:12.107448  2642 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.139461  2424 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:12.148228  2424 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.147841  2642 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.148803  2424 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:12.150848  2642 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.140863   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:12.141736  2652 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:12.143779  2653 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:12.089224   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.154945  2580 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:12.155231  2504 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:12.145303   926 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:13:12.128820   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 43969.6us
I20260709 13:13:12.157259   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.158057  2653 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:12.159317   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70209.9us
I20260709 13:13:12.160115  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.162150  1594 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:12.162526  2653 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:12.167022   847 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:12.167654   847 pending_rounds.cc:85] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 2
I20260709 13:13:12.167949   847 pending_rounds.cc:107] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:12.168206   847 raft_consensus.cc:2889] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:12.079888  2686 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.154456  2298 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.127905  2538 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.170679  2389 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:12.169266  2686 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.173612  2686 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.160140   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.175362   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.178694   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.186730  2686 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.186942   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.187637   851 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:12.187736   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.188326   855 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:12.189365   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.191221  2538 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:12.193651   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:12.194500  2686 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:12.194789  2345 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.194938  2686 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.195382  2686 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:12.171212  2399 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.174690  2546 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:12.203998   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:12.086557   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.207695   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.209014  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.212015  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.218271   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1390) took 1057 ms. Trace:
I20260709 13:13:12.225310   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.228636  2674 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.228843   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.229247  2674 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.229516  2424 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:12.229902   924 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1366) took 3826 ms (client timeout 2320 ms). Trace:
W20260709 13:13:12.230371   924 rpcz_store.cc:269] 0709 13:13:08.388976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.389103 (+   127us) service_pool.cc:224] Handling call
0709 13:13:09.710946 (+1321843us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800010598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:12.215331 (+2504385us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400080f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:12.215475 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5131888768}
I20260709 13:13:12.218837   998 rpcz_store.cc:276] 0709 13:13:11.160885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.259337 (+ 98452us) service_pool.cc:224] Handling call
0709 13:13:11.316966 (+ 57629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.588896 (+271930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:11.588931 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.588975 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:11.589007 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.217953 (+628946us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54000ea954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:12.218017 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.218196 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624850688}
I20260709 13:13:12.236074   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.236747   854 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:12.234305   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:12.239475   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79420.4us
I20260709 13:13:12.242339   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:12.243264   852 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:12.243645   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.245461   996 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:12.247340  2704 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.215179  2652 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.153493   927 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.254604   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.263870   998 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:12.240278  2580 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:12.233021  2573 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.231698  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.231583  2674 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.234921  2667 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:13:12.260830  2643 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:12.260063   749 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.234838  2669 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:12.305557   927 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.247953  2704 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.265306  1003 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:12.234153  2424 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.265657   852 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:12.269066  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.314822  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.317162  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.273419   849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:12.318568   849 raft_consensus.cc:606] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.265659   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 229) took 4920 ms. Trace:
I20260709 13:13:12.276448   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:12.321643   855 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.278470   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.281550   895 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.304749  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:12.255898   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 48336.3us
I20260709 13:13:12.271875   925 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:12.324666   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.269136  2546 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.300091   749 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.310343   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.309630   854 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:12.342105   749 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.344519  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.280942  2538 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:12.348325   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66871.8us
I20260709 13:13:12.349260   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:12.350075   998 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:12.351768   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.352648   930 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:12.358177   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.346195   749 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.310320  1596 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:12.269074  2707 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:13:12.369901   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.370818   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.374864   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.323871  2674 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.325625  2635 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.337962  2580 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:12.380800   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 7de58b0d5ad94d869fac3b0491143543 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51112
I20260709 13:13:12.381417  2543 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:12.338685  2306 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.347499  2310 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:12.309377  2643 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.385257   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.386451   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.435998   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.430361   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:12.439589   929 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:12.435832   924 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:12.442832   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.442867  1001 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:12.387832   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:12.392867   854 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:13:12.446739   854 pending_rounds.cc:85] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:12.447062   854 pending_rounds.cc:107] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:13:12.447402   854 raft_consensus.cc:2889] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:12.404904   851 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:12.379837   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:12.449446   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 69693.9us
I20260709 13:13:12.304430  2652 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:13:12.320739   751 rpcz_store.cc:276] 0709 13:13:07.394971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.398717 (+  3746us) service_pool.cc:224] Handling call
0709 13:13:12.255208 (+4856491us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:08.369011 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:08.413727 (+ 44716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:08.422777 (+  9050us) write_op.cc:435] Acquired schema lock
0709 13:13:08.424404 (+  1627us) tablet.cc:662] Decoding operations
0709 13:13:08.437052 (+ 12648us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:08.437176 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:08.437208 (+    32us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:13:08.438007 (+   799us) tablet.cc:701] Row locks acquired
0709 13:13:08.438055 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:13:08.438251 (+   196us) write_op.cc:270] Start()
0709 13:13:08.438418 (+   167us) write_op.cc:276] Timestamp: P: 1783602788438184 usec, L: 0
0709 13:13:08.438445 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:13:08.653702 (+215257us) log.cc:844] Serialized 3224 byte log entry
0709 13:13:09.490665 (+836963us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b54000db4a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab4dbe 00007f0d02abf930 00007f0d02abf8d8 00007f0d02abf858 00007f0d02abf813 00007f0d02abea9c 00007f0d089aa626 00007f0d089a4dd8 00007f0d02afb7cb 00007f0d02afe2c0
0709 13:13:09.664182 (+173517us) op_driver.cc:464] REPLICATION: finished
0709 13:13:09.789180 (+124998us) write_op.cc:301] APPLY: starting
0709 13:13:09.790321 (+  1141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:09.790572 (+   251us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:09.790591 (+    19us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:09.968914 (+178323us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:10.010127 (+ 41213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:13:10.010204 (+    77us) write_op.cc:312] APPLY: finished
0709 13:13:10.010615 (+   411us) log.cc:844] Serialized 65 byte log entry
0709 13:13:10.010803 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:10.010920 (+   117us) write_op.cc:454] Released schema lock
0709 13:13:10.010963 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:10.011307 (+   344us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:10.011337 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":114491,"apply.run_cpu_time_us":3896,"apply.run_wall_time_us":232295,"num_ops":8,"prepare.queue_time_us":277097,"prepare.run_cpu_time_us":28263,"prepare.run_wall_time_us":538842,"raft.queue_time_us":154149,"raft.run_cpu_time_us":26933,"raft.run_wall_time_us":57990,"replication_time_us":1225474,"spinlock_wait_cycles":13826432,"thread_start_us":384389,"threads_started":4,"wal-append.queue_time_us":198223,"wal-append.run_cpu_time_us":36390,"wal-append.run_wall_time_us":2139915}]]}
I20260709 13:13:12.353160  2642 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.376103  2649 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:12.444355   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:12.458912   998 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:12.376116  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:12.459877   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 72130.7us
I20260709 13:13:12.436287  2551 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.464141   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:12.447453   855 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:12.451327   926 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:12.370941   749 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:12.371909  2636 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:13:12.465876   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:12.455850   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:12.416694  2504 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.462088   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:12.469960   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:12.398531  2698 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
W20260709 13:13:12.455803   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.467392  1004 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:12.446895  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:12.481956   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.482406  1001 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:12.482661   921 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:12.408334  2674 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.481999  1004 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:12.479415   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.499372   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.457276  2680 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.509781  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:12.466925   749 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.518354   749 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:12.520392   749 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:12.516384  1001 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:12.527132   929 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:12.530907   849 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:12.430675  2543 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.468420  2705 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.504707   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.497923  2409 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:12.485394  2712 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:12.557911  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 323919us
I20260709 13:13:12.554381   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.535393  2527 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.559818   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.508469  2680 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.561595   855 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:12.478615  2666 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:13:12.538829  2638 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:12.484913  2504 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.552330  2705 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.580773  1566 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:12.567132   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:12.545053   749 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 3 to 4, term changed from 1 to 2, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4), VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.539100  2642 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:12.566458   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.584290   921 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:12.578926   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:12.551801  2658 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.547281  2670 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:12.652920   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:12.653811   921 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:12.633913 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:12.689751  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.571522  2527 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:12.586428  2538 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:12.633618  2389 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:12.693681  2717 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:13:12.710435  2298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.023s	sys 0.019s
W20260709 13:13:12.694844  2719 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:13:12.699586  2504 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.559932  2645 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.141s
I20260709 13:13:12.549696  2580 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:12.589758  2705 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.704625   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.772645   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:12.706118   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.652335  2642 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.577637  2680 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.709376  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:12.552652  2511 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:13:12.563268  2409 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:12.785559  2642 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.785725  2409 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.635347   749 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 4, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.577248  2712 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:13.029626  2712 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:13.016819  2721 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:13:12.762773   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:12.571496  2549 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.145s
I20260709 13:13:12.694518  2704 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:12.713135  2424 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.585634  2638 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:12.767037  2298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.024s	sys 0.019s
I20260709 13:13:12.776010   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.038403   928 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:13:12.707052  2686 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.000s
I20260709 13:13:12.765396  2710 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:12.784195  2504 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:12.788388  2511 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:12.767225  2389 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.048208  2638 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.798036   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.049247  2511 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:13.049839   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 251890us
I20260709 13:13:13.050163  2511 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:12.791087  2573 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:12.807900   921 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:12.828831   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:12.776686   922 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.030s	sys 0.055s
I20260709 13:13:12.797075  2674 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:12.803576  2666 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:12.812191  2713 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:12.861735   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.057186   925 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
W20260709 13:13:12.862948  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 282269us
W20260709 13:13:12.946715   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 405441us
I20260709 13:13:12.846182  2652 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 13:13:12.949553   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 349136us
I20260709 13:13:12.945418  2737 tablet_replica.cc:333] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:13:12.862947  2669 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:13.022557  2727 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:13:13.025411  2728 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 13:13:13.025532  2729 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:13.036499  2642 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.036931  2424 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.054804   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.054610   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.046273  2710 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:13.052524   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 223813us
I20260709 13:13:13.034276  1594 raft_consensus.cc:1217] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Deduplicated request from leader. Original: 1.4->[3.5-3.5]   Dedup: 3.5->[]
W20260709 13:13:13.122735   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71112.6us
I20260709 13:13:13.043529  2686 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.034519  2670 raft_consensus.cc:697] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:13.091982   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.051806  2573 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:13.056393   922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.030s	sys 0.055s
I20260709 13:13:13.045993  2527 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.043146  2726 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:13.049230  2389 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.063795  2345 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.059674  2649 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.063845  2667 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:13.099454  2713 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.101055  2511 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.092943  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.123528   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:13.169682   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46240.1us
I20260709 13:13:13.170532   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.125540  2580 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.129417  2041 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.171950  2041 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:13.134634   929 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:13.119237  2504 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.099921  2666 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.049232  2743 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.163710  2649 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:13.164402  2724 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:13:13.106040  2669 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.134337  1594 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:13:13.172544  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79696.4us
I20260709 13:13:13.121541  2642 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.121272  2749 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:13:13.173904  2345 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.175868   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:13.206149   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45196us
I20260709 13:13:13.207175   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.177263  2389 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:13.186677   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1368) took 1004 ms. Trace:
I20260709 13:13:13.203729  2298 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:13:13.209844  2580 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:13.209755   850 rpcz_store.cc:276] 0709 13:13:12.182549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.182686 (+   137us) service_pool.cc:224] Handling call
0709 13:13:12.224634 (+ 41948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.224983 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.225017 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.235210 (+ 10193us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.554310 (+319100us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:12.554351 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.554423 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.186289 (+631866us) raft_consensus.cc:1685] finished
0709 13:13:13.186317 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.186495 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2916,"raft.queue_time_us":497,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":124,"thread_start_us":292981,"threads_started":1,"wal-append.queue_time_us":583512}
I20260709 13:13:13.202950  1566 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.178874  2742 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:13.211652  2759 consensus_queue.cc:1059] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:13.212571  2759 raft_consensus.cc:3055] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:13.213022  2759 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:13.184477  2504 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.216511  2424 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:13.217310  2748 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:13:13.217772  2504 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.208195  2345 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.219676  2642 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.220098  2345 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.220201  2666 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:13.221204  2740 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:13.223425   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.224326  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.214043  2759 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.232758   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.234171   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.203209  2674 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:13.235550   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.236249  2389 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.246907  2389 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.248683   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:13.236670  2580 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:13.236374  2298 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.238333  2409 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.198025  2743 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.240556   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1428) took 1153 ms. Trace:
I20260709 13:13:13.222167  2747 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:13:13.241530  2504 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.245741  2759 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.224071  2670 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.243767  2674 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:13.276304  2754 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:13:13.260274  2686 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.258973  2756 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.259104  2751 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:13.259500  2750 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:13:13.269897  2567 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.307854  2750 consensus_queue.cc:1243] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:13.307981  2567 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.310155  2567 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.293859  2737 raft_consensus.cc:2243] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:13.311446  2737 raft_consensus.cc:2272] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:13.313256  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:13.314018  2768 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:13.314532  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.311458  2345 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.298841  2573 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.299458  2756 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.293135  2511 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:13.292236  2710 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:13.300628  2680 consensus_meta.cc:287] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.004s
I20260709 13:13:13.323505  2680 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:13.301703  2705 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 13:13:13.324697  2705 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.302529  2674 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.258606   922 rpcz_store.cc:276] 0709 13:13:12.083772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.113398 (+ 29626us) service_pool.cc:224] Handling call
0709 13:13:12.113868 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.114406 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.114455 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.115127 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.158124 (+1042997us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:13.163072 (+  4948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.173408 (+ 10336us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.222187 (+ 48779us) raft_consensus.cc:1685] finished
0709 13:13:13.223335 (+  1148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.236885 (+ 13550us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":180,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":1015903,"threads_started":1,"wal-append.queue_time_us":749381}
I20260709 13:13:13.303941  2424 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:13.304816   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:13.314780  2737 ts_tablet_manager.cc:1916] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:13.314879  2768 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.268790  2760 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:13.296734  2743 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.319916  2649 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.344002  2424 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:13.345064  2409 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:13.353283  2760 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:13.374024   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.374630  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.374969   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.375916   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.376788   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:13.377298   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 128711us
I20260709 13:13:13.345705  2750 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.389977   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.391445   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.392213   852 pending_rounds.cc:85] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 4
I20260709 13:13:13.393869   852 pending_rounds.cc:107] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:13:13.394160   852 raft_consensus.cc:2889] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:13.393693  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.396412  1601 raft_consensus.cc:2393] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:13.402675  2527 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:13.403012  2760 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.394937   855 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:13:13.397713  2763 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:13:13.392849  2750 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.389426  2409 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:13.394526  2511 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.416908  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.420817  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.408205  2680 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.428524   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.422670  1595 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:13.428180  2511 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.434764  2511 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.420279  2750 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.437069  2409 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.422065  2306 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:13.441761  2306 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.443951  2306 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.410763  2527 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.445574   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.446928  2527 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.419077  2345 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.444757  2424 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.449227  2424 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.449909  2713 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.451577   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 118) took 1067 ms. Trace:
I20260709 13:13:13.461640   849 rpcz_store.cc:276] 0709 13:13:12.384431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.389512 (+  5081us) service_pool.cc:224] Handling call
0709 13:13:12.530660 (+141148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.548828 (+ 18168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.451389 (+902561us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400071550. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:13.451519 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301647232}
I20260709 13:13:13.461733  2750 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.452497  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.463625  1596 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:13.463663   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1369) took 1249 ms. Trace:
I20260709 13:13:13.464298  2750 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.464212   848 rpcz_store.cc:276] 0709 13:13:12.216392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.224188 (+  7796us) service_pool.cc:224] Handling call
0709 13:13:12.259870 (+ 35682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.468658 (+208788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.478267 (+  9609us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:12.498145 (+ 19878us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:12.502477 (+  4332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.463356 (+960879us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54000ffd54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:13.463407 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.463542 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456795904}
I20260709 13:13:13.467085   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1301) took 1116 ms. Trace:
I20260709 13:13:13.452821  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.467662   927 rpcz_store.cc:276] 0709 13:13:12.350483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.356518 (+  6035us) service_pool.cc:224] Handling call
0709 13:13:12.370348 (+ 13830us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.548291 (+177943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.548332 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.556778 (+  8446us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.861174 (+304396us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:12.861482 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.466590 (+605108us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54002328d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:13.466659 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.466724 (+    65us) raft_consensus.cc:1685] finished
0709 13:13:13.466745 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.467027 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7461,"spinlock_wait_cycles":408377216,"thread_start_us":152395,"threads_started":1,"wal-append.queue_time_us":632544}
I20260709 13:13:13.473378  2543 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.459168  2573 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.490442   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.504235   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.510171   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.511468   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.512326   849 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:13.512775   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.518159   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.516388  2543 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:13.520444  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 130501us
W20260709 13:13:13.530050   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 294576us
W20260709 13:13:13.533428   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.535508  2543 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.536243  1596 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:13.536084  2543 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.537065  2712 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:13.537320  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.538587  2712 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.540704  2543 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.539155  2712 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.548398   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.590117   927 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:13.550426   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:13.595391   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.596088   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.553030   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.554489   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.597646   852 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:13.593997   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:13.598703   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:13.576783  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.601915   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.610661   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.614194   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:13.654280   927 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.521230  2680 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.645921  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 172) took 1378 ms. Trace:
I20260709 13:13:13.602695   851 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:13.650890   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.677801  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.514279  2508 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.659744  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.664057   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.678813  1003 rpcz_store.cc:276] 0709 13:13:12.267741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.331482 (+ 63741us) service_pool.cc:224] Handling call
0709 13:13:12.361402 (+ 29920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.361650 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:12.816829 (+455179us) log.cc:844] Serialized 14 byte log entry
0709 13:13:12.820141 (+  3312us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.614835 (+794694us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b18002c5a20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cff9ccc0a 00007f0d0910c219 00007f0d026f5d70 00007f0d026e9293 00007f0d026decea 00007f0d026df23a 00007f0d02b49a2a 00007f0d02b590bf 00007f0d02b59038
0709 13:13:13.645551 (+ 30716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.645606 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:13.645650 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.645722 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.645860 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35407,"prepare.queue_time_us":75335,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":6372,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":3517,"spinlock_wait_cycles":55073792,"thread_start_us":391422,"threads_started":1}
I20260709 13:13:13.680860   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:13.681849  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:13.664063   849 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:13.683247   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.650784  2543 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.692750  2737 ts_tablet_manager.cc:1929] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:13:13.652532  2514 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.686028  2743 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.704309   927 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:13.705137   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.706557   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.697717   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:13.685951  2686 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:13.714061   927 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:13.686350  2508 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.719480   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.712632  2686 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:13.712092  2514 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.722039  2737 log.cc:1199] T 7de58b0d5ad94d869fac3b0491143543 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/7de58b0d5ad94d869fac3b0491143543
I20260709 13:13:13.736701   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:13.728185  2686 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:13.709184  2543 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.774586  2543 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.775884  2772 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:13.769984  2699 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:13.776613  2772 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.777156  2699 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.777189   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.733372  2712 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.771338  2642 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:13.792922   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:13.794052  2737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51112 (request call id 75) took 1530 ms. Trace:
I20260709 13:13:13.785887  2759 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.785534  2758 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:13.785491  2772 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.795537  2642 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:13.802803  2758 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.807868  2758 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.808449  2642 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.809504  2514 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.797516  2737 rpcz_store.cc:276] 0709 13:13:12.338412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.380443 (+ 42031us) service_pool.cc:224] Handling call
0709 13:13:12.852317 (+471874us) ts_tablet_manager.cc:1151] Deleting tablet 7de58b0d5ad94d869fac3b0491143543
0709 13:13:12.858021 (+  5704us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:13:13.464481 (+606460us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:13.680308 (+215827us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:13.793965 (+113657us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180953,"tablet-delete.queue_time_us":395730,"thread_start_us":120212,"threads_started":1}
I20260709 13:13:13.801231  2759 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:13.815641  2759 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.825341   999 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:13.827744   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.828593   999 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:13.829420   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:13.832109   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:13.827131   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.826538  2514 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:13.826233  2538 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.834820  2652 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:13.860761  2652 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:13.829628  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 431799us
W20260709 13:13:13.893895   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 66898.6us
I20260709 13:13:13.836509   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:13.836316   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:13.924222   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 87843.8us
I20260709 13:13:13.940785  2773 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.941654  2773 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.928326  2514 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.948902  2514 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:13.948789  2742 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:13.928401  2543 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:13.950448  2726 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:13.950994  2726 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:13:13.953132  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 55193us
I20260709 13:13:13.951987  2726 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:13.892434  2508 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:14.192194   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 266621us
I20260709 13:13:13.841929  2713 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:14.031733  2726 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.079993  2773 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:14.195698  2713 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:14.196283  2713 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.105296  2643 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:14.108142   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1319) took 1405 ms. Trace:
I20260709 13:13:14.197721   929 rpcz_store.cc:276] 0709 13:13:12.702396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.709792 (+  7396us) service_pool.cc:224] Handling call
0709 13:13:14.107891 (+1398099us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400081454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:14.108063 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233719680}
I20260709 13:13:14.120492  2704 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:14.198915  2704 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:14.199820  2704 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.200826  2704 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.202999   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.863533  2652 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:13.838213  2699 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:13.861294  2686 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.210223   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:14.211838   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:13.963819  2538 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.269127  2686 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:14.276276  2538 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:14.279462  2756 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:14.294585   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 101174us
I20260709 13:13:14.322139  2705 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:14.326391   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:14.327095   848 raft_consensus.cc:1813] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Handling vote request from an unknown peer 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:14.326946   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:14.327867   848 raft_consensus.cc:2410] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:14.328855   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:14.329880   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:14.339483  2643 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.363451  2643 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.380003  2761 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.380576  2761 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:14.381270  2643 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:14.381569   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1323) took 1622 ms. Trace:
I20260709 13:13:14.381979   926 rpcz_store.cc:276] 0709 13:13:12.759741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.780635 (+ 20894us) service_pool.cc:224] Handling call
0709 13:13:12.780871 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.045694 (+264823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.045724 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.045769 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:13.045799 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.381336 (+1335537us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400445cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:14.381384 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.381513 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608575232}
I20260709 13:13:14.486147  2761 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:14.623471  2713 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.727742  2761 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.728446  2761 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:14.730859  2761 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:14.744719  2713 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:15.179459  2761 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:15.180229  2761 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:15.182674  2761 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:15.297847  2536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s	user 0.004s	sys 0.007s
W20260709 13:13:15.298399  2536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.004s	sys 0.008s
W20260709 13:13:15.426184   848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.003s	sys 0.002s
W20260709 13:13:15.426715   848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.003s	sys 0.002s
W20260709 13:13:15.845254   821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 2.006s (SENT) [suppressed 4 similar messages]
W20260709 13:13:16.080137   819 connection.cc:256] RPC call timeout handler was delayed by -1.9834355279999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
I20260709 13:13:16.100132  2656 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.100981  2656 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.103950  2656 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:16.104055   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 258819us
W20260709 13:13:16.137090   968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.240s	user 0.000s	sys 0.034s
W20260709 13:13:16.137564   968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.241s	user 0.000s	sys 0.034s
I20260709 13:13:16.138933  2345 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.139556  2345 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.141841  2345 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:16.145783  2504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.590s	user 0.018s	sys 0.056s
W20260709 13:13:16.146395  2504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.591s	user 0.019s	sys 0.057s
I20260709 13:13:16.153394  2345 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:16.154068  2345 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:16.156262  2345 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:16.158026  2504 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.158645  2504 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.160771  2504 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
W20260709 13:13:16.160964   784 debug-util.cc:398] Leaking SignalData structure 0x7b08001bb1e0 after lost signal to thread 32227
W20260709 13:13:16.162477   819 connection.cc:256] RPC call timeout handler was delayed by -1.2740547910000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:13:16.162959   819 connection.cc:256] RPC call timeout handler was delayed by -1.2255121230000441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 13:13:16.165060   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:16.165686   850 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:16.175539  2712 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:16.176327  2712 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.191417  2712 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:16.185424  2414 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.185091  2375 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.192380  2375 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.192314  2414 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.194795  2414 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:16.194916  2375 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:16.654275   970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.764s	user 0.015s	sys 0.066s
W20260709 13:13:16.654695   970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.764s	user 0.015s	sys 0.066s
W20260709 13:13:16.655742   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.657595  2414 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.658116  2414 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.660487  2414 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:16.666970  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 238) took 3901 ms. Trace:
W20260709 13:13:16.669894  2389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.882s	user 0.016s	sys 0.041s
W20260709 13:13:16.670339  2389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.887s	user 0.016s	sys 0.041s
I20260709 13:13:16.696658  1592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 239) took 3928 ms. Trace:
I20260709 13:13:16.697211  1592 rpcz_store.cc:276] 0709 13:13:12.769336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.770143 (+   807us) service_pool.cc:224] Handling call
0709 13:13:16.695764 (+3925621us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aeee 0000000000375939 00007f0d07a8e1e0 00007f0d07b570c4 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:16.695980 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.696215 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.696252 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:16.696312 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:16.696353 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.696432 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.696584 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4772736}
I20260709 13:13:16.667497  1595 rpcz_store.cc:276] 0709 13:13:12.765977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.766696 (+   719us) service_pool.cc:224] Handling call
0709 13:13:12.777019 (+ 10323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.431357 (+654338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.666648 (+3235291us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54001e2b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:16.666896 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457882368}
I20260709 13:13:16.824918  2712 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.825927  2712 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.868038  2389 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.883406  2389 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.886003  2389 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:16.887620  2580 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:16.888422  2580 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:16.890895  2712 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:16.891637  2712 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:16.892199  2712 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.937956  2414 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:16.938535  2389 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:16.938773  2414 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:16.939268  2389 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:16.941478  2414 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:16.941819  2389 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:17.010393  2580 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:17.435490   968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.278s	user 0.000s	sys 0.005s
W20260709 13:13:17.435992   968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.278s	user 0.000s	sys 0.005s
W20260709 13:13:17.436329   968 connection.cc:256] RPC call timeout handler was delayed by -1.1154552830000739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
I20260709 13:13:17.492856  2667 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.493856  2667 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:17.581086  2712 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:17.735613  2409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.283s	user 0.027s	sys 0.061s
W20260709 13:13:17.774891  2409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.322s	user 0.027s	sys 0.061s
I20260709 13:13:17.776461  2667 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:17.804397  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.005s	user 0.013s	sys 0.021s
W20260709 13:13:17.867511  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.013s	sys 0.021s
I20260709 13:13:17.905805  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:17.908710  1600 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:17.910751  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:17.911504  1600 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:17.914049  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:17.810128   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 371768us
W20260709 13:13:18.386885   851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.676s	user 0.016s	sys 0.046s
W20260709 13:13:18.389699   852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.700s	user 0.018s	sys 0.057s
W20260709 13:13:18.397634   852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.708s	user 0.020s	sys 0.058s
I20260709 13:13:18.399863   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1344) took 4914 ms. Trace:
I20260709 13:13:13.847563   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:18.379459   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1336) took 4952 ms. Trace:
I20260709 13:13:18.433710   847 rpcz_store.cc:276] 0709 13:13:13.428376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.429850 (+  1474us) service_pool.cc:224] Handling call
0709 13:13:13.441244 (+ 11394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:13.474371 (+ 33127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.474410 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:13.475170 (+   760us) log.cc:844] Serialized 52 byte log entry
0709 13:13:13.770251 (+295081us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:13.781136 (+ 10885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.784160 (+  3024us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.124131 (+339971us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b180039f2e0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:18.378347 (+4254216us) raft_consensus.cc:1685] finished
0709 13:13:18.378440 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.379091 (+   651us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32638,"raft.queue_time_us":35473,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2098,"spinlock_wait_cycles":62361984,"thread_start_us":257765,"threads_started":1,"wal-append.queue_time_us":262537,"wal-append.run_cpu_time_us":2025,"wal-append.run_wall_time_us":1002044}
W20260709 13:13:18.397400   851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.686s	user 0.019s	sys 0.048s
I20260709 13:13:18.457007  2696 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:18.469318  2696 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:18.457643   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:18.438647   854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1352) took 4600 ms (client timeout 2108 ms). Trace:
W20260709 13:13:18.490764   854 rpcz_store.cc:269] 0709 13:13:13.843750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.847196 (+  3446us) service_pool.cc:224] Handling call
0709 13:13:18.433491 (+4586295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:18.495306  2751 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.400385   852 rpcz_store.cc:276] 0709 13:13:13.485066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.679386 (+194320us) service_pool.cc:224] Handling call
0709 13:13:13.679585 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.679784 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:13.689124 (+  9340us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.398981 (+4709857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:18.399133 (+   152us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:18.399315 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.399454 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.399798 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11410,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"thread_start_us":4709113,"threads_started":1}
I20260709 13:13:18.903015  2504 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:18.904809  2504 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:18.946877   967 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 5.051s	user 0.044s	sys 0.081s
W20260709 13:13:18.947448   967 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 5.052s	user 0.046s	sys 0.081s
W20260709 13:13:18.947916   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 5.05192e+06us
I20260709 13:13:18.949441   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:19.010780   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 61708.1us
I20260709 13:13:19.012871   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:19.145267  2768 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:19.174906  2768 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:19.191675  2768 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:19.267423  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.081s	sys 0.154s
W20260709 13:13:19.267921  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.082s	sys 0.154s
I20260709 13:13:19.284096  2751 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:19.329345  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.402650  2772 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:19.403239  2772 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:19.403697  2772 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.432443  2696 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:19.436729  2751 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:19.437568  2751 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:19.438508  2751 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:19.449826  2642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 5.617s	user 0.033s	sys 0.033s
W20260709 13:13:19.450383  2642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 5.621s	user 0.034s	sys 0.034s
I20260709 13:13:19.463392  2829 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:19.488075  2829 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:19.627617  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 388) took 6315 ms. Trace:
I20260709 13:13:19.628206  1597 rpcz_store.cc:276] 0709 13:13:13.312392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.312506 (+   114us) service_pool.cc:224] Handling call
0709 13:13:13.312877 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.394833 (+ 81956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.396920 (+  2087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.400751 (+  3831us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:13.405015 (+  4264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.626884 (+6221869us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b5400122fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:19.626991 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.627293 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167256960}
I20260709 13:13:19.631860  1000 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:19.632225  2799 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:13:19.682132  1601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 6.141s	user 0.023s	sys 0.049s
W20260709 13:13:19.682726  1601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 6.142s	user 0.023s	sys 0.049s
W20260709 13:13:19.691457   922 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 5.891s	user 0.019s	sys 0.022s
W20260709 13:13:19.691992   922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 5.893s	user 0.021s	sys 0.022s
I20260709 13:13:19.695811   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1307) took 7270 ms. Trace:
I20260709 13:13:19.696458   930 rpcz_store.cc:276] 0709 13:13:12.417253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.417460 (+   207us) service_pool.cc:224] Handling call
0709 13:13:12.417786 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.560243 (+142457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.560273 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:12.560313 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:12.560339 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.687127 (+7126788us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400444dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:19.687227 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.687491 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327154048}
I20260709 13:13:13.844620   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:19.698575   849 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1353) took 5854 ms (client timeout 2027 ms). Trace:
W20260709 13:13:19.699239   849 rpcz_store.cc:269] 0709 13:13:13.844214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.844353 (+   139us) service_pool.cc:224] Handling call
0709 13:13:19.698353 (+5854000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:19.700651   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1310) took 7250 ms. Trace:
I20260709 13:13:19.701179   924 rpcz_store.cc:276] 0709 13:13:12.450394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.450668 (+   274us) service_pool.cc:224] Handling call
0709 13:13:12.451011 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.464845 (+ 13834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.464875 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:12.464915 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:12.464946 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.699938 (+7234992us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54004470d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:19.700021 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.700434 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31401728}
I20260709 13:13:19.647979  2504 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:19.730245  2751 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:19.731042   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 718366us
I20260709 13:13:19.737948   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:19.743770  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1415) took 6198 ms. Trace:
I20260709 13:13:19.745924  1002 rpcz_store.cc:276] 0709 13:13:13.544640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.547894 (+  3254us) service_pool.cc:224] Handling call
0709 13:13:13.548283 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.548633 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.548663 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.549156 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.830740 (+281584us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:13.830791 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.830868 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.743147 (+5912279us) raft_consensus.cc:1685] finished
0709 13:13:19.743184 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.743430 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":987,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":281178,"threads_started":1,"wal-append.queue_time_us":6152219}
I20260709 13:13:19.717660  2504 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:19.763744  2804 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:19.763233  2829 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.763772  2504 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:19.801419 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:19.780570  2824 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:19.857311  2824 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.859748  2669 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:19.860471  2669 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:19.857393  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1427) took 6180 ms. Trace:
I20260709 13:13:19.861289  2669 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.865571  2669 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:19.870045   855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 6.379s	user 0.044s	sys 0.049s
I20260709 13:13:19.872375  2829 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.868170  2824 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:19.862520  1003 rpcz_store.cc:276] 0709 13:13:13.676917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.684633 (+  7716us) service_pool.cc:224] Handling call
0709 13:13:13.686534 (+  1901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.730173 (+ 43639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.730213 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.730263 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:13.730292 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.787324 (+6057032us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b54002972d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:19.819022 (+ 31698us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.857196 (+ 38174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99791104}
I20260709 13:13:19.881117   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 127) took 6490 ms. Trace:
W20260709 13:13:19.885115  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.887655   855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 6.397s	user 0.046s	sys 0.049s
W20260709 13:13:19.868445   821 connection.cc:256] RPC call timeout handler was delayed by -2.3017887549999614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 13:13:19.888057   846 rpcz_store.cc:276] 0709 13:13:13.390487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.392235 (+  1748us) service_pool.cc:224] Handling call
0709 13:13:13.392493 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.396876 (+  4383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.396905 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.401752 (+  4847us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.796951 (+395199us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:13.796990 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.797052 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.131365 (+334313us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4800002dd8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:19.880446 (+5749081us) raft_consensus.cc:1685] finished
0709 13:13:19.880490 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.880865 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4040,"raft.queue_time_us":4216,"raft.run_cpu_time_us":2865,"raft.run_wall_time_us":733723,"spinlock_wait_cycles":5273472,"thread_start_us":374952,"threads_started":1,"wal-append.queue_time_us":6466776}
I20260709 13:13:19.849905  2772 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.924763  2825 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:19.931928   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.931773  2825 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.959548  2825 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:19.966172   784 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3660 after lost signal to thread 787
I20260709 13:13:19.965782  2772 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:19.981873  2828 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:19.982769  2828 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:19.984268  2828 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:19.932202  2817 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:19.988306  2817 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:19.995750   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1333) took 6335 ms. Trace:
I20260709 13:13:19.996433   922 rpcz_store.cc:276] 0709 13:13:13.659490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.677310 (+ 17820us) service_pool.cc:224] Handling call
0709 13:13:13.699554 (+ 22244us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:13.724567 (+ 25013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.725651 (+  1084us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:13.795105 (+ 69454us) log.cc:844] Serialized 52 byte log entry
0709 13:13:19.692896 (+5897791us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:19.692995 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.693202 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.994436 (+301234us) raft_consensus.cc:1685] finished
0709 13:13:19.994494 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.994801 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4971,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":4763,"spinlock_wait_cycles":45824,"thread_start_us":5895727,"threads_started":1,"wal-append.queue_time_us":6196924}
I20260709 13:13:20.012585  2817 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.024458   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1346) took 6533 ms. Trace:
I20260709 13:13:20.025480   851 rpcz_store.cc:276] 0709 13:13:13.491470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.681815 (+190345us) service_pool.cc:224] Handling call
0709 13:13:13.684598 (+  2783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.684957 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.684987 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.689831 (+  4844us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.456627 (+4766796us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:18.456703 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.485352 (+ 28649us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.022413 (+1537061us) raft_consensus.cc:1685] finished
0709 13:13:20.022443 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.024372 (+  1929us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7887,"thread_start_us":4745127,"threads_started":1,"wal-append.queue_time_us":6311403}
I20260709 13:13:20.032593  2833 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.038064   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 126) took 6648 ms. Trace:
W20260709 13:13:20.038532   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.040050   969 connection.cc:256] RPC call timeout handler was delayed by -4.6259437460000754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:13:20.038654   855 rpcz_store.cc:276] 0709 13:13:13.390108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.390267 (+   159us) service_pool.cc:224] Handling call
0709 13:13:13.390583 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.394825 (+  4242us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:13.490500 (+ 95675us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.913060 (+6422560us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.928410 (+ 15350us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.931135 (+  2725us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:19.931187 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.931406 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.037749 (+106343us) raft_consensus.cc:1685] finished
0709 13:13:20.037769 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.037994 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3872,"raft.queue_time_us":9398,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":1767,"spinlock_wait_cycles":13978880,"thread_start_us":6398818,"threads_started":1,"wal-append.queue_time_us":6529273}
W20260709 13:13:20.040820   969 connection.cc:256] RPC call timeout handler was delayed by -4.61442069099985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 13:13:20.041335   969 connection.cc:256] RPC call timeout handler was delayed by -4.5505206909999742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 13:13:20.041719   969 connection.cc:256] RPC call timeout handler was delayed by -4.3605220310000732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 13:13:20.041901  2817 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:20.042107   969 connection.cc:256] RPC call timeout handler was delayed by -4.3459810309998375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
W20260709 13:13:20.042570   969 connection.cc:256] RPC call timeout handler was delayed by -4.2054356670000743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 13:13:20.043077   969 connection.cc:256] RPC call timeout handler was delayed by -3.8656540509998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28900000000000003s for a response
I20260709 13:13:20.043165  2828 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.044051   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:20.044345  2828 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.043077  2805 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.045924  2805 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.047116  2835 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:20.048036  2835 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.048910  2805 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.050721  2828 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.042608  2817 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.040016  2833 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.053489  2817 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.054082  2817 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:20.054689  2833 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.054759  2817 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.055541  2833 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.055989   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.057337  2817 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.058737  2835 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.067056   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1396) took 7820 ms. Trace:
I20260709 13:13:20.068068  2817 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:20.056439  2833 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.067919  2828 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:20.067580   996 rpcz_store.cc:276] 0709 13:13:12.246813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.251801 (+  4988us) service_pool.cc:224] Handling call
0709 13:13:12.252128 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.269887 (+ 17759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.269933 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.305531 (+ 35598us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.790913 (+485382us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:12.790955 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.066284 (+7275329us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54003e3354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:20.066360 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.066466 (+   106us) raft_consensus.cc:1685] finished
0709 13:13:20.066480 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.066851 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38176,"raft.queue_time_us":1587,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":19974,"spinlock_wait_cycles":37472512,"thread_start_us":418080,"threads_started":1,"wal-append.queue_time_us":518712,"wal-append.run_cpu_time_us":2024,"wal-append.run_wall_time_us":1168025}
I20260709 13:13:20.078189  2817 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.076897  2533 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:20.080966  2817 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.081377  2533 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.084414  2533 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:20.068266   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.088551   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.089385   851 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.089363   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.090364   855 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.093781  2833 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.097958   851 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.098816  2836 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:20.099500  2836 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.102505   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.102957  2836 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.103653   846 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.103713  2836 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:20.105145  2836 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.108152  2836 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.108598   855 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:20.111342   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.113418   846 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:20.123832   784 debug-util.cc:398] Leaking SignalData structure 0x7b080003dc00 after lost signal to thread 879
W20260709 13:13:20.124218   784 debug-util.cc:398] Leaking SignalData structure 0x7b080003dbe0 after lost signal to thread 880
I20260709 13:13:20.125254   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.126330   855 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.128774   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.129725   851 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.134562   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.136344   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.138054   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.140190   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.140831   849 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.141753   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.142616   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.143054   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.143221   855 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:20.144359   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.144675   846 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:20.144984   852 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.145273   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.146008   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.149129  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:20.153348   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 5.85597e+06us
I20260709 13:13:20.154912   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:20.156347   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.158576   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.158761   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.159206  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 338) took 6369 ms. Trace:
I20260709 13:13:20.159427   846 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.189306   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.190026   846 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:20.190887   928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 6.371s	user 0.009s	sys 0.075s
I20260709 13:13:20.190970   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.159439   849 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.160389   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.162137   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:20.163463   819 connection.cc:256] RPC call timeout handler was delayed by -2.5575590559999455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:13:20.164394  2808 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:20.166410  2805 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:20.167308  2807 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:20.168426   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.171705   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:20.191457   928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 6.372s	user 0.009s	sys 0.075s
W20260709 13:13:20.210654   819 connection.cc:256] RPC call timeout handler was delayed by -2.0220590559999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:13:20.216884   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45310.2us
I20260709 13:13:20.219486  2828 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.220065  2828 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.220631  2828 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.222885  2818 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:20.179457   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.224570   847 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.224454  2803 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:20.226410   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.228782   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.211771  1596 rpcz_store.cc:276] 0709 13:13:13.789604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.791540 (+  1936us) service_pool.cc:224] Handling call
0709 13:13:13.802630 (+ 11090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.959972 (+6157342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.960026 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.960081 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:19.960139 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.158822 (+198683us) spinlock_profiling.cc:243] Waited 6.15 s on lock 0x7b54006f6154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:20.158873 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.159026 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14155890560}
I20260709 13:13:20.237844   969 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.238844  2804 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.239033   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:20.239409  2804 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.239981  2804 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:20.202128   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47404.3us
I20260709 13:13:20.212527   855 raft_consensus.cc:2376] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4. Re-sending same reply.
I20260709 13:13:20.204310  2649 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.242771  2803 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.243369  2803 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.243409   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:20.244019  2803 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.178267   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.213440  2808 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.247591  2808 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.248304  2504 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.251052  2504 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.256572  2804 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.235916  2828 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.263406   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.269472   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:20.269605  2504 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.270133  2504 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.270272  2828 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.270675  2504 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.265560  2804 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.271018   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:20.272275  2818 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.272795  2818 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.273211  2818 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.274544  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:20.275712  1593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 237) took 7524 ms (client timeout 2096 ms). Trace:
W20260709 13:13:20.276111  1593 rpcz_store.cc:269] 0709 13:13:12.751367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.759552 (+  8185us) service_pool.cc:224] Handling call
0709 13:13:20.274203 (+7514651us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:20.275544 (+  1341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6457088}
I20260709 13:13:20.280146  2504 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.280279  2803 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.282673  2504 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.282811  2803 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.282902  2768 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.287493  2828 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.288106  2828 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.288383   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.288708  2828 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.293255  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 335) took 6759 ms. Trace:
I20260709 13:13:20.290499  2504 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.293704  1601 rpcz_store.cc:276] 0709 13:13:13.533833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.534086 (+   253us) service_pool.cc:224] Handling call
0709 13:13:13.534537 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.534947 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.534986 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.535506 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.692261 (+6156755us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:19.692457 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.692606 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.292973 (+600367us) raft_consensus.cc:1685] finished
0709 13:13:20.292992 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.293192 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27,"raft.queue_time_us":257,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":6142578,"threads_started":1,"wal-append.queue_time_us":6451724}
I20260709 13:13:20.283198  2818 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.299232  2504 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.302937  2818 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:20.303918  2815 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.277s	user 0.000s	sys 0.003s
I20260709 13:13:20.305848   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1361) took 5979 ms. Trace:
I20260709 13:13:20.306365   848 rpcz_store.cc:276] 0709 13:13:14.326641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.329783 (+  3142us) service_pool.cc:224] Handling call
0709 13:13:14.330151 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.331134 (+   983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.331352 (+   218us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.370298 (+ 38946us) log.cc:844] Serialized 26 byte log entry
0709 13:13:15.427205 (+1056907us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:15.427263 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.427369 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.303060 (+4875691us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:20.305249 (+  2189us) raft_consensus.cc:1685] finished
0709 13:13:20.305280 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.305635 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":273,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":13420800,"thread_start_us":1042242,"threads_started":1,"wal-append.queue_time_us":5626956}
I20260709 13:13:20.315182   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.316275  2504 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.322005   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.322894   855 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:20.334832  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 229137us
W20260709 13:13:20.342872   784 debug-util.cc:398] Leaking SignalData structure 0x7b08000d96e0 after lost signal to thread 954
I20260709 13:13:20.334782  2828 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:20.344812   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd780 after lost signal to thread 1028
W20260709 13:13:20.350472  1566 connection.cc:256] RPC call timeout handler was delayed by -4.9081937340001787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:13:20.347489   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd740 after lost signal to thread 1029
I20260709 13:13:20.351758   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.351904  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.352933  2768 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:20.353235   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.346576  2828 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.354460   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.353366  2768 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.354732   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.356737   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.355208   851 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:20.357824   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.355175  2768 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:20.361085   853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1351) took 6524 ms (client timeout 1765 ms). Trace:
W20260709 13:13:20.362600   923 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 6.678s	user 0.043s	sys 0.062s
W20260709 13:13:20.362907   923 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 6.679s	user 0.043s	sys 0.062s
W20260709 13:13:20.361493   853 rpcz_store.cc:269] 0709 13:13:13.836956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.360754 (+6523798us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:20.360899 (+   145us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:20.376034   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1414) took 6831 ms. Trace:
I20260709 13:13:20.376715   995 rpcz_store.cc:276] 0709 13:13:13.544306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.548136 (+  3830us) service_pool.cc:224] Handling call
0709 13:13:13.548421 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.548739 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.548765 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.549231 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.833970 (+284739us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:13.834016 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.834082 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.375492 (+6541410us) raft_consensus.cc:1685] finished
0709 13:13:20.375537 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.375831 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1138,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":11648,"thread_start_us":284365,"threads_started":1,"wal-append.queue_time_us":6469061}
I20260709 13:13:20.381299  2768 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.401875  2768 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.421381  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 177) took 7074 ms. Trace:
I20260709 13:13:20.422057  1000 rpcz_store.cc:276] 0709 13:13:13.347253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.347456 (+   203us) service_pool.cc:224] Handling call
0709 13:13:13.351673 (+  4217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.632555 (+6280882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.420903 (+788348us) spinlock_profiling.cc:243] Waited 6.27 s on lock 0x7b540035b454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:20.421281 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14409762176}
I20260709 13:13:20.436061   855 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:20.440537   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.443785   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.487469   846 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.444082   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.447098   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.496446   851 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.487828   852 raft_consensus.cc:1240] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:20.488746   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.502938  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:20.504952  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 80308.3us
W20260709 13:13:20.507627   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.505179   847 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.509094   853 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:20.514303   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.516690   855 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.518581   969 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:20.551529   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.554767  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 115585us
I20260709 13:13:20.579267   846 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.580628   851 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:20.580858   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.582077   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.582976  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:20.587885  2826 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:20.590037  2826 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.590468  2826 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.591836  2409 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:20.592906  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.593225  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:20.589061  2803 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.596511  2803 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.597704  2803 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.600023  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:20.600688  1593 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.600860  1596 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:20.601732  2827 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:20.602351  2827 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.603243  2805 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:20.603549  2827 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.604076  2805 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.588414  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.613838  1000 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:20.600806  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:20.615794   853 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:20.620592  1003 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:20.619172   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.619227   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.623999   846 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.623934  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.629778  2673 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:20.630512  2673 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:20.631808  2673 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.635108  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:20.642315  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:20.643738   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.655956  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:20.656874  1596 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:20.659318   855 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.665715  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:20.667186  2751 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:20.671077   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.671316  2818 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.671993  2772 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.672024  2818 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.672672  2772 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.673677  2772 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.674466  2818 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.676589  2804 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:20.677304  2804 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.677978  2826 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.678746   855 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:20.679157   855 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:20.681763   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:20.675424   967 connection.cc:256] RPC call timeout handler was delayed by -4.5480084770001667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3227s for a response
I20260709 13:13:20.667488  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:20.685920  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:20.693531   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:20.694381   851 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.695053  1593 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.680227   855 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.685305  2751 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:20.696245  2409 raft_consensus.cc:697] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:20.700835   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003ccea0 after lost signal to thread 1625
W20260709 13:13:20.706269   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003cce60 after lost signal to thread 1626
I20260709 13:13:20.679535  2804 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:20.709182  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.709565  1003 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:20.701625   855 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.710112  1000 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:20.702443  2707 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:20.703259  2851 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:13:20.713411   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.680562  2826 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:20.730942   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53998.3us
I20260709 13:13:20.731817  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:20.735081   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:20.704877  2751 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.737457  2751 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:20.737866  2409 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.736039  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:20.744462   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 46963.9us
I20260709 13:13:20.745358  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.746019   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.746236  2345 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.716055  2768 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:20.746213   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:20.746171  1003 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:20.754622   846 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:20.746685  2345 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.753435   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:20.755565  2345 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.763515  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.764827  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.763597  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.766594  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.767177  1003 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:20.765388  1001 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:20.768344   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.771363   851 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:20.755798  2856 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:20.776888   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.776880  2751 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:20.778512   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.789604  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.788704  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.792948   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.784140  2772 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:20.795393   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003825a0 after lost signal to thread 2716
W20260709 13:13:20.795897   784 debug-util.cc:398] Leaking SignalData structure 0x7b080030c660 after lost signal to thread 2717
W20260709 13:13:20.796262   784 debug-util.cc:398] Leaking SignalData structure 0x7b0800348700 after lost signal to thread 2722
W20260709 13:13:20.796535   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003487e0 after lost signal to thread 2723
I20260709 13:13:20.784405  2856 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:20.790865   855 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:20.805836   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.798333  2673 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:20.813216  2673 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:20.814229  2673 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.799489   847 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:20.803994  2856 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:20.811722  2760 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:20.856658   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95312.9us
I20260709 13:13:20.865499   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.872870   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.882014   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.885807   849 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:20.884869   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.887445   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:20.888401   996 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:20.881486   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.878854  2803 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.891752  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.889176   998 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:20.899062  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.904804  2862 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:20.905639  2862 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.878705  2827 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:13:20.898276   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:20.908401   847 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:20.895642  2772 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.913865   854 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:20.915201   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:20.925639   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.895824  2803 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:20.939719  2643 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.959218  2803 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:20.960001  2772 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.913785  2751 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:20.961220  2853 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:20.942832   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:20.971177  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:20.973378  2863 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:20.973922  2863 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.974339  2863 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.906953  2862 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:20.981711   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.984074  2805 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:20.984851   750 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:20.987939  2846 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.303s	user 0.002s	sys 0.001s
I20260709 13:13:20.957644  2409 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 13:13:20.992302   951 debug-util.cc:398] Leaking SignalData structure 0x7b080030eca0 after lost signal to thread 32227
I20260709 13:13:20.959259  2643 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:20.993425   951 debug-util.cc:398] Leaking SignalData structure 0x7b080030e5a0 after lost signal to thread 787
W20260709 13:13:20.994805   951 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7ce0 after lost signal to thread 879
W20260709 13:13:20.995100   951 debug-util.cc:398] Leaking SignalData structure 0x7b080005bfa0 after lost signal to thread 880
I20260709 13:13:20.970880  2345 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:20.997687  2345 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:20.999181   951 debug-util.cc:398] Leaking SignalData structure 0x7b080003fae0 after lost signal to thread 954
W20260709 13:13:21.000329   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6ec0 after lost signal to thread 1028
W20260709 13:13:21.000775   951 debug-util.cc:398] Leaking SignalData structure 0x7b0800311660 after lost signal to thread 1029
W20260709 13:13:21.002087   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0e20 after lost signal to thread 1625
W20260709 13:13:21.002364   951 debug-util.cc:398] Leaking SignalData structure 0x7b080015dca0 after lost signal to thread 1626
W20260709 13:13:21.005853   951 debug-util.cc:398] Leaking SignalData structure 0x7b0800326d20 after lost signal to thread 2424
W20260709 13:13:21.006126   951 debug-util.cc:398] Leaking SignalData structure 0x7b08001dfb00 after lost signal to thread 2508
I20260709 13:13:21.006523  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.017020   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.013828  2707 consensus_queue.cc:1059] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:13:21.030058  2707 raft_consensus.cc:3055] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:21.030490  2707 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:21.031562  2643 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.032867  2707 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.038362  1002 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:21.039175   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:21.048594  2854 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.346s	user 0.000s	sys 0.007s
I20260709 13:13:21.032994  2856 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.066553  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.067333  1000 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:21.068845  1002 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:20.307807   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 7de58b0d5ad94d869fac3b0491143543 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
W20260709 13:13:21.070004   740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 762385us
I20260709 13:13:20.884750 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:21.067371   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.085546   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.084518  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:21.087381   999 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:21.089504  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:21.090277  1593 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:21.086544  2856 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.093338   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.096539   996 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:21.101080  2760 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:21.103000  2760 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:21.105137   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.084820  2707 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.103472  2760 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:21.099918  1598 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.124729   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:21.127625   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.132462  2667 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:21.132985  2667 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.133401  2752 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:21.134001  2752 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.134881  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.135860  2863 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.138037  2863 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:21.138489  2752 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:21.145304  1004 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:21.139070   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 3.227s (SENT) [suppressed 20 similar messages]
I20260709 13:13:21.148293   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:21.148475  2760 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.148969   854 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:21.149598  2760 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:21.150826   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003aef40 after lost signal to thread 2652
W20260709 13:13:21.151362   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1e60 after lost signal to thread 2653
W20260709 13:13:21.151839   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd580 after lost signal to thread 2686
W20260709 13:13:21.152220   951 debug-util.cc:398] Leaking SignalData structure 0x7b0800259720 after lost signal to thread 2708
W20260709 13:13:21.152514   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd4e0 after lost signal to thread 2727
W20260709 13:13:21.152823   951 debug-util.cc:398] Leaking SignalData structure 0x7b08001007e0 after lost signal to thread 2739
W20260709 13:13:21.153061   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd540 after lost signal to thread 2743
W20260709 13:13:21.153347   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd5a0 after lost signal to thread 2746
W20260709 13:13:21.153610   951 debug-util.cc:398] Leaking SignalData structure 0x7b08001b57e0 after lost signal to thread 2753
W20260709 13:13:21.153823   951 debug-util.cc:398] Leaking SignalData structure 0x7b080031da60 after lost signal to thread 2756
I20260709 13:13:21.133414  2667 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:21.148337  2698 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:21.162638  2686 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:21.166639   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.169838   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.176546   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:21.177431   852 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:21.179179   853 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:21.164901  2807 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:21.167044  2698 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:21.189729   750 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:21.198274   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 954985us
I20260709 13:13:21.199501  2640 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.199958  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.200274  2640 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.200763  2652 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:21.201336  2652 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:21.202116   894 connection.cc:256] RPC call timeout handler was delayed by -4.2767860869998913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 13:13:21.202548   894 connection.cc:256] RPC call timeout handler was delayed by -1.2630247929998859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.5356s for a response
I20260709 13:13:21.171970  2808 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:21.193446  2698 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:21.208942  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:21.209844  1003 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:21.210894   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1332) took 7551 ms. Trace:
I20260709 13:13:21.214015  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:21.214733   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 75698.7us
I20260709 13:13:21.209407  2538 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:21.202972  2640 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.249476   854 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:21.218775   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 138) took 7902 ms. Trace:
I20260709 13:13:21.209733  2649 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.253859  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.254508   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.216477  2841 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.209441  2514 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:21.218504  2543 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:21.267411   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54391.9us
I20260709 13:13:21.251008   928 rpcz_store.cc:276] 0709 13:13:13.316018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.316514 (+   496us) service_pool.cc:224] Handling call
0709 13:13:13.316819 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:13.342797 (+ 25978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.342826 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:13.343801 (+   975us) log.cc:844] Serialized 52 byte log entry
0709 13:13:13.819176 (+475375us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b180042ba40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:13:20.208527 (+6389351us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:20.208936 (+   409us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.204150 (+995214us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.204337 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.209646 (+  5309us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.212298 (+  2652us) raft_consensus.cc:1685] finished
0709 13:13:21.212328 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.218717 (+  6389us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":994759,"prepare.queue_time_us":2379,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":1041528,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":28517,"spinlock_wait_cycles":9740416,"thread_start_us":6388998,"threads_started":1,"wal-append.queue_time_us":6408675}
I20260709 13:13:21.253508  2424 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.196002  2686 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.268414  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.211261   923 rpcz_store.cc:276] 0709 13:13:13.659115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.659656 (+   541us) service_pool.cc:224] Handling call
0709 13:13:13.659934 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.660124 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:20.363290 (+6703166us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.191918 (+828628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.194947 (+  3029us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:21.197683 (+  2736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.205115 (+  7432us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.210835 (+  5720us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":659776,"prepare.queue_time_us":6083553,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":679610,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":6836573,"threads_started":2,"wal-append.queue_time_us":209384}
I20260709 13:13:21.273895  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.216898  2808 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:21.275125   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56840.1us
I20260709 13:13:21.210049  2807 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.295207   851 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:21.295231  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.296170  1601 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 1.
I20260709 13:13:21.218523  2538 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.299526  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.290987  2680 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.293128  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.293537  2573 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:21.292338  2652 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.311300  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.319872  2743 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.275624  2649 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.313277  2807 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.315045  2808 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.320505  2743 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.322118  2649 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.271378  2841 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.322847  2743 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.323426  2536 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:21.324154  2841 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.324059  2536 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.326704  2536 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.294864  2424 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:21.328452  2872 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.253s	user 0.004s	sys 0.000s
I20260709 13:13:21.328580  2686 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.332211  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:21.334606   895 connection.cc:256] RPC call timeout handler was delayed by -4.2536428549999528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:13:21.336763  2820 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:21.338344  2820 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:21.338889  2820 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:21.338001   895 connection.cc:256] RPC call timeout handler was delayed by -4.0484428549998484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:13:21.342329   895 connection.cc:256] RPC call timeout handler was delayed by -3.9773428549999608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 13:13:21.343257   895 connection.cc:256] RPC call timeout handler was delayed by -3.9737428549999549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
I20260709 13:13:21.309517  2052 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:21.351769   895 connection.cc:256] RPC call timeout handler was delayed by -3.9539428549999229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260709 13:13:21.351758  1598 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:21.352514   895 connection.cc:256] RPC call timeout handler was delayed by -3.8142984719997912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 13:13:21.353933   895 connection.cc:256] RPC call timeout handler was delayed by -3.8126428549999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
W20260709 13:13:21.354240   895 connection.cc:256] RPC call timeout handler was delayed by -3.739742854999804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 13:13:21.354522   895 connection.cc:256] RPC call timeout handler was delayed by -3.7361428550000255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
I20260709 13:13:21.356320   851 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:21.354987  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.359825  2772 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:21.360451  2818 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.360953  2818 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.361346   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:21.362439  2877 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:21.362902  2877 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:21.362947  2818 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.363948  2877 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.360489  2772 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.338099  2424 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.366971  2772 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:21.367683   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:21.352986  2841 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.368896  2841 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.370846  2841 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.358650  2686 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.372219  2686 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.374490  2686 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.377580  2680 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.379801  2680 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.355239  2667 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.381285   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:21.382158  2667 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.384043  2052 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.389700  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.389977  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.390432  1595 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:21.392011  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.392876  1597 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:21.393903  2653 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:21.394726  2538 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.310588  2573 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.396150   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 90a3708334cc4bd59c73d494d2747d54 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42376
I20260709 13:13:21.396406  1593 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.397410  2573 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:21.290927  2514 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.403568  2514 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.405634  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.403818   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 116) took 10583 ms. Trace:
I20260709 13:13:21.394611  2653 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:21.383425   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:21.355816  2743 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:21.351939  2760 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:13:21.408226   749 rpcz_store.cc:276] 0709 13:13:10.823808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.824015 (+   207us) service_pool.cc:224] Handling call
0709 13:13:21.401093 (+10577078us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:13.225418 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:13.226227 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:13.226272 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:13:13.226299 (+    27us) tablet.cc:662] Decoding operations
0709 13:13:13.227776 (+  1477us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:13.227878 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:13.227919 (+    41us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:13:13.228955 (+  1036us) tablet.cc:701] Row locks acquired
0709 13:13:13.228982 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:13:13.229184 (+   202us) write_op.cc:270] Start()
0709 13:13:13.229333 (+   149us) write_op.cc:276] Timestamp: P: 1783602793229122 usec, L: 0
0709 13:13:13.229371 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:13:13.230281 (+   910us) log.cc:844] Serialized 4016 byte log entry
0709 13:13:13.539868 (+309587us) op_driver.cc:464] REPLICATION: finished
0709 13:13:13.865694 (+325826us) write_op.cc:301] APPLY: starting
0709 13:13:13.865808 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:13.866099 (+   291us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:13.886597 (+ 20498us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:13.887880 (+  1283us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:13.888100 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:13:13.888147 (+    47us) write_op.cc:312] APPLY: finished
0709 13:13:13.955884 (+ 67737us) log.cc:844] Serialized 77 byte log entry
0709 13:13:14.021075 (+ 65191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:14.047825 (+ 26750us) write_op.cc:454] Released schema lock
0709 13:13:14.049163 (+  1338us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:14.070909 (+ 21746us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:14.073777 (+  2868us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2173243,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":308613,"apply.run_cpu_time_us":16821,"apply.run_wall_time_us":200460,"num_ops":10,"prepare.queue_time_us":126138,"prepare.run_cpu_time_us":21169,"prepare.run_wall_time_us":31585,"raft.queue_time_us":135339,"raft.run_cpu_time_us":20677,"raft.run_wall_time_us":226032,"replication_time_us":310322,"thread_start_us":443667,"threads_started":4,"wal-append.queue_time_us":43295,"wal-append.run_cpu_time_us":6083,"wal-append.run_wall_time_us":1739729}]]}
I20260709 13:13:21.417455   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.418270   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.420707   922 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:21.422057   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:21.515280   930 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.422858   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:21.516368   924 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.422948   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.423451   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.423971   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.415282  2052 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:21.426124   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.426997   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.424664   852 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:21.429741   854 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:21.434162  2680 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:21.434216  2536 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.436868  2829 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:21.438314  2863 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:21.438935  2504 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:13:21.441668  2667 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:21.433626   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50368.7us
I20260709 13:13:21.437978  2862 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 13:13:21.557701   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 269011us
I20260709 13:13:21.466492  2573 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.558691   921 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:21.476435  1593 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:21.477523  1596 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:13:21.474155  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:21.564469  1598 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:21.503438   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.513408   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.517722   926 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:21.528671   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.531745  2536 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.535749  2666 raft_consensus.cc:2764] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:21.525803  2680 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:21.577780  2680 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.543865  2805 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.582389  2805 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.544492   925 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:21.545178   929 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:21.550802   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 345825us
I20260709 13:13:21.584738   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.586344   996 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.435027  2743 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.588018   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:21.589334   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.589231  2686 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:21.446693  2698 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:21.589634   998 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.589990  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:21.590329  2686 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:21.570695   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.567880   924 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:21.591907  2643 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:21.592366  2643 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:21.592839  2643 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.593595   923 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:21.589337  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:21.594498   999 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.593724   996 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.589677  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.591264  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.564116  1596 pending_rounds.cc:85] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 5
I20260709 13:13:21.558673  2667 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:21.591935   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.590752  2686 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:21.618598  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:21.619855   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:21.622960   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:21.623588   927 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.624637  1003 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:21.601068  1002 raft_consensus.cc:2368] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:21.631844   930 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:21.635251  1001 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:21.519558  2820 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:21.657812   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:21.642025   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50156.2us
I20260709 13:13:21.660710   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:21.667203   854 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:21.591789  2698 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:21.685556  2828 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:13:21.567205  2818 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:13:21.627166  1596 pending_rounds.cc:107] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:13:21.591787   922 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:21.592628  2656 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.158s
I20260709 13:13:22.135099  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.072144  1002 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:22.071334   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.147195  1596 raft_consensus.cc:2889] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.792335  2707 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.357s
W20260709 13:13:22.079648   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:22.012526   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 183651us
I20260709 13:13:22.151480   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.152585  2656 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.152993  2656 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:22.134718   851 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:13:21.752866  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52766.3us
I20260709 13:13:21.701834   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.753681  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.146559   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:22.176708   818 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.041s	sys 0.067s
W20260709 13:13:22.190524   818 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.041s	sys 0.075s
W20260709 13:13:22.190922   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 626169us
I20260709 13:13:22.185168   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:21.749398  1565 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.120.62:45937, user_credentials={real_user=slave}} blocked reactor thread for 63299.8us
I20260709 13:13:22.200620   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.203843   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:21.824878  2667 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.205554   930 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:22.212496   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:21.665434  2891 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:21.703437  2856 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:21.686841  2696 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:21.779515  2514 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.223973  2696 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.230384  2696 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:22.230731  2888 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:13:22.231400  2667 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:22.224845  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.232259  1001 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
W20260709 13:13:22.233886   927 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.000s
I20260709 13:13:22.234525   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.211136   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:21.821322  2686 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:22.153802  2656 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.431824   849 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:22.298099  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.440457   927 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:13:22.296751   969 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.049s	sys 0.032s
W20260709 13:13:22.441900   969 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.049s	sys 0.039s
I20260709 13:13:22.315317  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.443333  1003 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:22.304541   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.326139   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.328984   853 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:22.306313   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.314746   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.449893   851 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:22.281621   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.450973  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.452018  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.306962   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.453019   924 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:22.299242   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.235426   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.395516  2893 tablet_replica.cc:333] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:13:22.467729   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1372) took 1017 ms. Trace:
I20260709 13:13:22.455120   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:22.268983   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.468009   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.316186  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:22.455565   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1356) took 1027 ms. Trace:
I20260709 13:13:22.472671   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:22.386757  2892 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:13:22.395853  2894 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:13:22.471221   921 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:22.477885   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.473668   925 rpcz_store.cc:276] 0709 13:13:21.450342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.285453 (+835111us) service_pool.cc:224] Handling call
0709 13:13:22.467634 (+182181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.481101   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1368) took 1025 ms. Trace:
I20260709 13:13:22.474638   924 rpcz_store.cc:276] 0709 13:13:21.428104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.269577 (+841473us) service_pool.cc:224] Handling call
0709 13:13:22.455492 (+185915us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.541498   926 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:22.271476  2820 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:22.542317   923 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:22.222692  2891 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:22.312572  2805 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:22.222483  2856 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.481125   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1379) took 1021 ms. Trace:
I20260709 13:13:22.544238   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1365) took 1096 ms. Trace:
I20260709 13:13:22.235168  2536 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:22.551030   929 rpcz_store.cc:276] 0709 13:13:21.448069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.284798 (+836729us) service_pool.cc:224] Handling call
0709 13:13:22.544179 (+259381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.546113   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.554030   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1363) took 1106 ms. Trace:
I20260709 13:13:22.547344   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.550782   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1352) took 1119 ms. Trace:
I20260709 13:13:22.481503   927 rpcz_store.cc:276] 0709 13:13:21.449062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.467714 (+1018652us) service_pool.cc:224] Handling call
0709 13:13:22.474279 (+  6565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.554999   923 rpcz_store.cc:276] 0709 13:13:21.426932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.188356 (+761424us) service_pool.cc:224] Handling call
0709 13:13:22.546379 (+358023us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.552191   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1355) took 1117 ms. Trace:
I20260709 13:13:22.556593   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.472219  2667 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.550858   921 rpcz_store.cc:276] 0709 13:13:21.460059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.282644 (+822585us) service_pool.cc:224] Handling call
0709 13:13:22.481056 (+198412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.560446   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.561488   924 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:22.556511   928 rpcz_store.cc:276] 0709 13:13:21.427810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.194238 (+766428us) service_pool.cc:224] Handling call
0709 13:13:22.545418 (+351180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.554409   930 rpcz_store.cc:276] 0709 13:13:21.447400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.471881 (+1024481us) service_pool.cc:224] Handling call
0709 13:13:22.553966 (+ 82085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.384892  2878 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.163s
I20260709 13:13:22.278806  2514 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.571280   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.545183  2743 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:22.574460   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.583447   921 raft_consensus.cc:2410] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:13:22.575177   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.551589  2656 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.553077  2856 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.581589   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.582319   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.589468   923 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:22.591558   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1354) took 1164 ms. Trace:
I20260709 13:13:22.592101   927 rpcz_store.cc:276] 0709 13:13:21.427491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.555444 (+1127953us) service_pool.cc:224] Handling call
0709 13:13:22.591488 (+ 36044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.587994  2643 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.994s	user 0.010s	sys 0.002s
I20260709 13:13:22.585363   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.601568   930 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:22.547083  2891 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.589437  2820 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.599330   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.590744   925 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:22.588650   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.605681   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1353) took 1178 ms. Trace:
I20260709 13:13:22.588912  2504 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:22.607981   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.605314   921 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:22.608953   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1347) took 1183 ms. Trace:
I20260709 13:13:22.584044   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1351) took 1135 ms. Trace:
I20260709 13:13:22.609277   921 rpcz_store.cc:276] 0709 13:13:21.425463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.590596 (+1165133us) service_pool.cc:224] Handling call
0709 13:13:22.608894 (+ 18298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.587096  2696 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.611083   924 rpcz_store.cc:276] 0709 13:13:21.426612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.552854 (+1126242us) service_pool.cc:224] Handling call
0709 13:13:22.561962 (+  9108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.606037   925 rpcz_store.cc:276] 0709 13:13:21.427201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.556323 (+1129122us) service_pool.cc:224] Handling call
0709 13:13:22.605615 (+ 49292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.570915  2667 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:22.615442   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.615705   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:22.616518   819 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.040s	sys 0.118s
I20260709 13:13:22.616880   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:22.619908   819 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.040s	sys 0.118s
I20260709 13:13:22.619649   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.619917   850 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
W20260709 13:13:22.620740   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.620606   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.617322   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.622052   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.623059   927 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:22.623948   852 raft_consensus.cc:2368] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:22.599942  2828 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:22.617702  2504 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:22.604432  2643 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:22.631887   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.639256   819 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:22.639853   927 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:22.637620   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.611843  2820 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:22.636158   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.637140   925 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 1.
I20260709 13:13:22.643030   853 raft_consensus.cc:2393] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:22.648571   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:22.649420   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1358) took 1208 ms. Trace:
I20260709 13:13:22.649756   930 rpcz_store.cc:276] 0709 13:13:21.440886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.638071 (+1197185us) service_pool.cc:224] Handling call
0709 13:13:22.649360 (+ 11289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.650899   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.657822   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.661712   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:22.660481   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:22.656898  2409 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.061s
W20260709 13:13:22.666010  2409 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.014s	sys 0.061s
I20260709 13:13:22.666826  2667 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:22.668229   854 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.640774  2666 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.628204  2828 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.670212   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.667320  2820 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:22.670583   925 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:22.676343   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.678426   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.678766   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.679132   850 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:22.681254   930 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:22.681944   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1378) took 1222 ms. Trace:
I20260709 13:13:22.682314   930 rpcz_store.cc:276] 0709 13:13:21.459819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.650450 (+1190631us) service_pool.cc:224] Handling call
0709 13:13:22.681876 (+ 31426us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.686570  2896 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:13:22.687048   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.688159   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1348) took 1262 ms. Trace:
I20260709 13:13:22.688445   930 rpcz_store.cc:276] 0709 13:13:21.425775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.684961 (+1259186us) service_pool.cc:224] Handling call
0709 13:13:22.688048 (+  3087us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.684983   927 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:22.688563   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1474) took 1054 ms. Trace:
I20260709 13:13:22.695818   848 rpcz_store.cc:276] 0709 13:13:21.634455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.634685 (+   230us) service_pool.cc:224] Handling call
0709 13:13:21.662385 (+ 27700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.662802 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:13:21.662834 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.663357 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.450512 (+787155us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:22.450570 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.450632 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.688315 (+237683us) raft_consensus.cc:1685] finished
0709 13:13:22.688334 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.688488 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128982,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":15422,"thread_start_us":357262,"threads_started":1,"wal-append.queue_time_us":953215}
I20260709 13:13:22.698725   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.700233   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1350) took 1273 ms. Trace:
I20260709 13:13:22.701226   923 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:22.695894  2900 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.701467   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1361) took 1255 ms. Trace:
I20260709 13:13:22.708304   925 rpcz_store.cc:276] 0709 13:13:21.441753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.696376 (+1254623us) service_pool.cc:224] Handling call
0709 13:13:22.696716 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.696908 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.696936 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.696987 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:22.697024 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.697095 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.697209 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.714859   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 242254us
I20260709 13:13:22.716985   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.715561   925 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:22.702911   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1367) took 1254 ms. Trace:
W20260709 13:13:22.719833   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.700896   930 rpcz_store.cc:276] 0709 13:13:21.426315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.689634 (+1263319us) service_pool.cc:224] Handling call
0709 13:13:22.690715 (+  1081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.699773 (+  9058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.699814 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.699875 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:22.699924 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.700006 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.700142 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.719884   923 rpcz_store.cc:276] 0709 13:13:21.448767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.700444 (+1251677us) service_pool.cc:224] Handling call
0709 13:13:22.700923 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.702568 (+  1645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.702833 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.723793   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 62177.7us
I20260709 13:13:22.724655   819 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:22.727926   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1369) took 1270 ms. Trace:
I20260709 13:13:22.732750   925 rpcz_store.cc:276] 0709 13:13:21.449379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.712430 (+1263051us) service_pool.cc:224] Handling call
0709 13:13:22.712850 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.718907 (+  6057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.719279 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.736359   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.735437   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:22.737182   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.738281   925 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:22.739378   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:22.740084   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.740322   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:22.740779   925 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.744506   930 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:22.741101   925 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:22.746976   819 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:22.763414   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1374) took 1286 ms. Trace:
I20260709 13:13:22.764117   930 rpcz_store.cc:276] 0709 13:13:21.458667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.722222 (+1263555us) service_pool.cc:224] Handling call
0709 13:13:22.736832 (+ 14610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.745128 (+  8296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.745376 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.764490   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.765782   926 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:22.791903   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1360) took 1350 ms. Trace:
I20260709 13:13:22.734797  2573 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:22.736289  2514 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:22.802008   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73183.9us
W20260709 13:13:22.744415  2680 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.166s	user 0.011s	sys 0.002s
I20260709 13:13:22.760995  2643 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:22.779942   925 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.905248   925 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.739152  2904 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.748260  2698 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:22.748410  2667 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.899432   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:22.899833  2907 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:22.909443  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 345500us
I20260709 13:13:22.921912  2907 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:22.922672  2907 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:22.914263   853 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.924086  2904 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:22.924620   851 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.792367   926 rpcz_store.cc:276] 0709 13:13:21.441447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.269498 (+828051us) service_pool.cc:224] Handling call
0709 13:13:22.791830 (+522332us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.902779   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 170240us
W20260709 13:13:22.900864   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.907294  2818 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:22.910838   930 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.932408   930 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:22.920676   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:22.741839  2829 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 13:13:22.927385   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:22.936178   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:22.802039  2673 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:13:22.921446  2698 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:22.743510  2504 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.905529  2751 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:22.919793  2889 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:22.921018  2828 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:22.858843  2680 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.931680  2514 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:22.931967  2820 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:22.924871  2904 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:23.061780   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.115646   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 216331us
I20260709 13:13:23.117486   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:23.094612  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 162279us
I20260709 13:13:22.938509  2878 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.119570   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1357) took 1638 ms. Trace:
I20260709 13:13:23.120020   927 rpcz_store.cc:276] 0709 13:13:21.440514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.691523 (+1251009us) service_pool.cc:224] Handling call
0709 13:13:22.691899 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.703955 (+ 12056us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:22.900453 (+196498us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.078368 (+177915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.078416 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:23.078465 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.078535 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.078671 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":338114,"prepare.run_wall_time_us":55,"raft.queue_time_us":1907,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":100,"spinlock_wait_cycles":80885504,"thread_start_us":211763,"threads_started":2}
I20260709 13:13:23.124230  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.124598   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.125679   922 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.020013   930 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:22.937879  2818 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.160485  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.162081  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.119246  2665 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:23.163234   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.120059  2698 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:23.131840   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.132457   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.136356  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:23.136884  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.136829  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.136206  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:23.146435  2856 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.003s
I20260709 13:13:23.127072  2916 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:23.125175  2899 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:23.153517  2673 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:23.158602   925 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:23.238381   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1376) took 1779 ms. Trace:
I20260709 13:13:23.238739   925 rpcz_store.cc:276] 0709 13:13:21.459238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.740413 (+1281175us) service_pool.cc:224] Handling call
0709 13:13:22.740664 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.238108 (+497444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.238325 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.163923   922 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:23.164481  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.164821   930 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.160190   927 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:23.172092   926 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:23.240392   927 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:23.240760   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1382) took 1779 ms. Trace:
I20260709 13:13:23.236682  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.241108   926 rpcz_store.cc:276] 0709 13:13:21.460884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.120700 (+1659816us) service_pool.cc:224] Handling call
0709 13:13:23.127868 (+  7168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.240511 (+112643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.240701 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.241240   927 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.120635  2514 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.246098  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:23.122221  2751 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:23.244854  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 113377us
W20260709 13:13:23.245373   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.120762  2820 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:23.236660   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 104253us
I20260709 13:13:23.246825  1593 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 1.
I20260709 13:13:23.248589   927 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:23.173748  2911 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:13:23.254725   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:23.261746  2656 consensus_meta.cc:287] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.005s
I20260709 13:13:23.262648   850 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:23.264418   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.252462   925 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:23.265487   925 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:23.265836  1600 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:23.269798   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 141) took 1659 ms. Trace:
I20260709 13:13:23.270218   926 rpcz_store.cc:276] 0709 13:13:21.610897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.242904 (+1632007us) service_pool.cc:224] Handling call
0709 13:13:23.252211 (+  9307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.269457 (+ 17246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.269487 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.269533 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.269572 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.269632 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.269745 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.235067  2665 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.245066  2829 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.281150  1594 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:23.288023  2907 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.288444   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.275518   925 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.294770   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 155) took 1606 ms. Trace:
I20260709 13:13:23.296380  2917 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.298115  1564 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:23.313750   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.349386  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51407.9us
I20260709 13:13:23.308326   924 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:23.350879   930 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:23.318025   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 142) took 1693 ms. Trace:
I20260709 13:13:23.351112   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:23.237177  2856 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.319990   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:23.308542   927 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:23.352442   924 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:23.270368  2828 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.307241  2829 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:23.353968   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 89618.5us
I20260709 13:13:23.345880   996 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:23.308455   925 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.347432   854 rpcz_store.cc:276] 0709 13:13:21.688188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.688339 (+   151us) service_pool.cc:224] Handling call
0709 13:13:21.696563 (+  8224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.668680 (+972117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.294453 (+625773us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b540043d354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:23.294690 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219477248}
I20260709 13:13:23.350816  1564 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.247423  2899 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:23.351460   926 rpcz_store.cc:276] 0709 13:13:21.611217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.271493 (+1660276us) service_pool.cc:224] Handling call
0709 13:13:23.271910 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.304308 (+ 32398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.304379 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.304443 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.304503 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.304589 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.304765 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.246848  2673 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.373003  2673 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.361387   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.362401   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1383) took 1817 ms. Trace:
I20260709 13:13:23.373831   927 rpcz_store.cc:276] 0709 13:13:21.535070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.159415 (+1624345us) service_pool.cc:224] Handling call
0709 13:13:23.160033 (+   618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.352533 (+192500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.352711 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.375084   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:23.375720   927 raft_consensus.cc:2376] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3. Re-sending same reply.
I20260709 13:13:23.376272  2904 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.376593   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:23.376808  2829 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.378396   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 143) took 1766 ms. Trace:
I20260709 13:13:23.378774   927 rpcz_store.cc:276] 0709 13:13:21.611504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.377538 (+1766034us) service_pool.cc:224] Handling call
0709 13:13:23.377839 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.378057 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.378083 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.378126 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.378166 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.378223 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.378342 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.362663   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.333225  2862 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:23.362802  2828 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.369680  2919 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:23.370365   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.368918  2856 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:23.365911   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 58166.6us
W20260709 13:13:23.390564   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.380983   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:23.381341   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.375741  1592 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:23.249768  2916 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.395587   926 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:23.396010   926 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:23.396078   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.394613  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.383658  2743 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:23.399037  2743 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.398550  2899 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.382377   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1377) took 1892 ms. Trace:
I20260709 13:13:23.385373   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:23.364012  2901 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:23.399312  2891 consensus_meta.cc:287] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.000s
I20260709 13:13:23.412348   930 rpcz_store.cc:276] 0709 13:13:21.459530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.804534 (+1345004us) service_pool.cc:224] Handling call
0709 13:13:22.910658 (+106124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.351395 (+440737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.351624 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.386312  2828 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.383970   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.413185   846 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:23.421844   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.385842  2862 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.384549  2914 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:23.422902   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.423105  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:23.400386  2920 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:13:23.399477  2743 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:23.429832  2891 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.400017  2696 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.430501  2538 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:23.386528  2919 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.409380   926 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.436304   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.353525  2917 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.422570   855 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:23.435165  2862 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:23.387015  2673 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:23.402942  2514 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.422940  2901 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:23.444461   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60549.1us
I20260709 13:13:23.439388  2918 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.446038   846 raft_consensus.cc:2410] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:23.447692   925 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:23.467760   926 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.448871  2919 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.434338  2680 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.440009  1595 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:23.433713  2929 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:23.464131   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 66546.5us
I20260709 13:13:23.468374   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1384) took 1875 ms. Trace:
I20260709 13:13:23.470396   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:23.475991   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.481611   925 rpcz_store.cc:276] 0709 13:13:21.593328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.239462 (+1646134us) service_pool.cc:224] Handling call
0709 13:13:23.248146 (+  8684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.468104 (+219958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.468312 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.482936   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.484282  2907 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:23.485791  2820 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:23.486228   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.486672   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.487555  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.487931  1564 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.488375   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.489116   853 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:23.489239  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.458374  2818 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:23.492841   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.469502  2901 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:23.496927   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.497419   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1450) took 1291 ms. Trace:
I20260709 13:13:23.500056   925 rpcz_store.cc:276] 0709 13:13:22.206374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.482599 (+1276225us) service_pool.cc:224] Handling call
0709 13:13:23.482917 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.496961 (+ 14044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.496992 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.497043 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.497092 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.497159 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.497319 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.467465  2918 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:23.503458   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.509213   926 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:23.510437   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1385) took 1822 ms. Trace:
I20260709 13:13:23.511029   926 rpcz_store.cc:276] 0709 13:13:21.687754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.387017 (+1699263us) service_pool.cc:224] Handling call
0709 13:13:23.387319 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.510021 (+122702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.510354 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.513883   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1451) took 1241 ms. Trace:
W20260709 13:13:23.515725   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.514487   925 rpcz_store.cc:276] 0709 13:13:22.262666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.502781 (+1240115us) service_pool.cc:224] Handling call
0709 13:13:23.503155 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.503496 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.503776 (+   280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.503839 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.503893 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.503955 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.504079 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.518875   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.498517  2820 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:23.528306   925 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:23.513446  2918 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.498726  2673 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.532347   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.534926  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.564675   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:23.495967  2514 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:13:23.532033  2818 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:23.551488  2891 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.560410  2916 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:23.562251   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.569419   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:23.571997  2538 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:13:23.584612   849 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:23.585980   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1366) took 1380 ms. Trace:
I20260709 13:13:23.592811   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.592547   852 rpcz_store.cc:276] 0709 13:13:22.206003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.206514 (+   511us) service_pool.cc:224] Handling call
0709 13:13:23.585641 (+1379127us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400128ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:23.585860 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351624064}
I20260709 13:13:23.529444  2665 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.611699   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1454) took 1035 ms. Trace:
I20260709 13:13:23.612360   925 rpcz_store.cc:276] 0709 13:13:22.576179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.573003 (+996824us) service_pool.cc:224] Handling call
0709 13:13:23.573411 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.573619 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.573643 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.611239 (+ 37596us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:23.611315 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.611413 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.611619 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:23.621373   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 87254.3us
I20260709 13:13:23.622853   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.626703   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1447) took 1598 ms. Trace:
I20260709 13:13:23.627321   930 rpcz_store.cc:276] 0709 13:13:22.028157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.421882 (+1393725us) service_pool.cc:224] Handling call
0709 13:13:23.431475 (+  9593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.431944 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.431984 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.432669 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.571399 (+138730us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.571462 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.571570 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.626412 (+ 54842us) raft_consensus.cc:1685] finished
0709 13:13:23.626443 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.626622 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37961,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":7440,"thread_start_us":138263,"threads_started":1,"wal-append.queue_time_us":191278}
I20260709 13:13:23.513751  2680 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:13:23.570343  2696 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.563014  2673 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.630532   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1446) took 1824 ms. Trace:
I20260709 13:13:23.637552   924 rpcz_store.cc:276] 0709 13:13:21.821600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.395012 (+1573412us) service_pool.cc:224] Handling call
0709 13:13:23.395349 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.410509 (+ 15160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.410536 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.435846 (+ 25310us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.624274 (+188428us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.624325 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.624411 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.630077 (+  5666us) raft_consensus.cc:1685] finished
0709 13:13:23.630116 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.630455 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15293,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11281,"thread_start_us":166715,"threads_started":1,"wal-append.queue_time_us":181190}
I20260709 13:13:23.640722   924 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:23.641556   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 157) took 1054 ms. Trace:
I20260709 13:13:23.644912   924 rpcz_store.cc:276] 0709 13:13:22.587375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.639905 (+1052530us) service_pool.cc:224] Handling call
0709 13:13:23.640331 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.641234 (+   903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.641484 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:23.642338  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 151756us
I20260709 13:13:23.635846   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:23.621898  2856 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:23.647264  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.649202   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1455) took 1050 ms. Trace:
I20260709 13:13:23.651918   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:23.651738   924 rpcz_store.cc:276] 0709 13:13:22.598720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.645900 (+1047180us) service_pool.cc:224] Handling call
0709 13:13:23.648072 (+  2172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.648364 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.648411 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.648469 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.648520 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.648590 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.649134 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.644824  2573 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.655591   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 155) took 1068 ms. Trace:
I20260709 13:13:23.637044  2673 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.657049   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:23.656739   930 rpcz_store.cc:276] 0709 13:13:22.586737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.629280 (+1042543us) service_pool.cc:224] Handling call
0709 13:13:23.647289 (+ 18009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.655116 (+  7827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.655166 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.655231 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.655295 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.655383 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.655516 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.575302  2829 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.660815   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.661985   855 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:23.668640   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:23.669417   849 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:23.651044  2856 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:23.676714   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1457) took 1045 ms. Trace:
I20260709 13:13:23.636408  2696 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:23.678799   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.682610   921 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:23.680838   924 rpcz_store.cc:276] 0709 13:13:22.638413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.653559 (+1015146us) service_pool.cc:224] Handling call
0709 13:13:23.660901 (+  7342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.676344 (+ 15443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.676378 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.676425 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.676462 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.676520 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.676642 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.656248  2573 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:23.668148  2805 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:23.693993   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 144) took 2007 ms. Trace:
I20260709 13:13:23.694491   927 rpcz_store.cc:276] 0709 13:13:21.686899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.380879 (+1693980us) service_pool.cc:224] Handling call
0709 13:13:23.381320 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.381552 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:23.480349 (+ 98797us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.693623 (+213274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.693679 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.693735 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.693807 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.693938 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133196,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":254661,"threads_started":3}
I20260709 13:13:23.703043  2665 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:23.684587  2856 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:23.704577   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 1.09256e+06us
W20260709 13:13:23.704568  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57389.1us
I20260709 13:13:23.689982  2573 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.669548  2698 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.686080  2772 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:13:23.689975  2514 consensus_queue.cc:1243] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:23.736796  2696 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.687431  2929 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:23.738207  2743 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.741487  2573 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.742805   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 150) took 1397 ms. Trace:
I20260709 13:13:23.679288  2914 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:13:23.720613  2680 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:23.749718   818 connection.cc:256] RPC call timeout handler was delayed by -1.2557995039999241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
I20260709 13:13:23.756795   854 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:23.758042   926 rpcz_store.cc:276] 0709 13:13:22.345471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.512309 (+1166838us) service_pool.cc:224] Handling call
0709 13:13:23.512752 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.515538 (+  2786us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:23.530133 (+ 14595us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.742435 (+212302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.742491 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:23.742541 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.742611 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.742741 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14404,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":8227,"thread_start_us":129164,"threads_started":2}
W20260709 13:13:23.759738   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.758234   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65140.9us
W20260709 13:13:23.764840   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.765745   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.763937  2680 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.752318  2932 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:13:23.776322  2680 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.778420  2828 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.779244   851 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:23.759646  2937 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:23.779974   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.780238  2829 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:23.780679   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.750133  2918 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.784370  2573 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:23.784284  2901 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:23.785238   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1379) took 1095 ms. Trace:
I20260709 13:13:23.786446  2917 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.766009  2914 consensus_queue.cc:1243] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:23.792245   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:23.817261   850 rpcz_store.cc:276] 0709 13:13:22.689508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.698425 (+  8917us) service_pool.cc:224] Handling call
0709 13:13:23.785044 (+1086619us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000c9ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:23.785177 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251295104}
I20260709 13:13:23.794005  2919 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:23.798130  2940 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:23.824946   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.798986  2751 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:23.798846   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.752678  2514 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:23.805531   921 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:23.807103   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131688us
I20260709 13:13:23.814404  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.829440   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.816403   851 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:23.826100   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.832584   924 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:23.833461   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.834466   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.835188  2409 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:23.835898  2573 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:23.836663  2409 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:23.836967  2907 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:23.837545  1592 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:23.838124   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.838340   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:23.828621   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 264003us
I20260709 13:13:23.835814  2829 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:23.839447   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.840180   928 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:23.840342  2829 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:23.840754  2829 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:23.841192   928 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:23.841583  2829 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.842406   928 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:23.842705  2829 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.839473   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:23.844045  2862 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:23.844537  2893 raft_consensus.cc:2243] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:23.844846  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:23.836606  2772 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 13:13:23.826633   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.835510   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.848166   924 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:23.819681  2698 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:23.862907   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:23.863857  2862 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:23.879642   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.865244  2743 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:24.163280   928 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:23.896442   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.176326   927 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:23.932144   926 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:23.934953   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:23.865397  2901 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:23.868670  2914 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:13:23.939766  2538 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:23.934062  2828 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:23.963361   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.831238  2514 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:23.827229  2863 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:24.207048  2863 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:24.162999  2878 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.041s	user 0.026s	sys 0.003s
W20260709 13:13:24.082442   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 243049us
I20260709 13:13:24.082799  1003 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:24.083146  1004 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:23.860061  2899 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:24.083590  1002 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.084450   846 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:24.086790   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.100260  2924 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:24.090226  2906 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:24.082172   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 132574us
I20260709 13:13:24.093080   925 raft_consensus.cc:3055] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:13:24.094796  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 334555us
I20260709 13:13:24.099925   923 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:13:24.102028   922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1364) took 2635 ms (client timeout 1775 ms). Trace:
I20260709 13:13:23.887288   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:24.164232   818 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.506s (SENT)
W20260709 13:13:24.164939   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.103058   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.180018   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1494) took 1051 ms. Trace:
I20260709 13:13:24.181151  1598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 368) took 1535 ms. Trace:
I20260709 13:13:24.083678  2931 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:24.084147  2944 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:24.205792  2673 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.008s
I20260709 13:13:24.219360   997 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.095548  2696 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:24.224162   998 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:24.088065   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 203) took 1156 ms. Trace:
I20260709 13:13:24.178508  2698 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.207412  2514 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:24.430372  1000 consensus_meta.cc:287] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.033s
I20260709 13:13:24.425894  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.431465   925 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:24.315640   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.431931  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:24.207504  2863 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:24.416738  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 241483us
I20260709 13:13:24.180444  2666 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.435043   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.432704   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:24.457095   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.457913  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:24.458386   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:24.459815   925 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.435145   850 rpcz_store.cc:276] 0709 13:13:23.129010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.953534 (+824524us) service_pool.cc:224] Handling call
0709 13:13:24.179442 (+225908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.179692 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.179732 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.179776 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:24.179807 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.179862 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.179964 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.430910  1000 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:24.435804   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:24.430120   922 rpcz_store.cc:269] 0709 13:13:21.447716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.151188 (+703472us) service_pool.cc:224] Handling call
0709 13:13:23.890459 (+1739271us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b5400444dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:24.083530 (+193071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238759168}
I20260709 13:13:24.434666   996 rpcz_store.cc:276] 0709 13:13:22.931350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.931555 (+   205us) service_pool.cc:224] Handling call
0709 13:13:22.952835 (+ 21280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.354921 (+402086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.087758 (+732837us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54000eb354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:24.087972 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903340800}
W20260709 13:13:24.435781   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.463390   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.435431   921 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:24.436121   924 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:24.452790  2900 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:24.465659  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:24.455124   967 connection.cc:256] RPC call timeout handler was delayed by -1.762283848999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:13:24.466647   967 connection.cc:256] RPC call timeout handler was delayed by -1.3599838489999456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 13:13:24.435508  1598 rpcz_store.cc:276] 0709 13:13:22.628258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.628385 (+   127us) service_pool.cc:224] Handling call
0709 13:13:22.637668 (+  9283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.666400 (+ 28732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.666429 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.666850 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.761622 (+ 94772us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:22.761685 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.889696 (+1128011us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b54001e2b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:24.082490 (+192794us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.082549 (+    59us) raft_consensus.cc:1685] finished
0709 13:13:24.082566 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.163735 (+ 81169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4940,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4713,"spinlock_wait_cycles":65201920,"thread_start_us":94386,"threads_started":1,"wal-append.queue_time_us":266939,"wal-append.run_cpu_time_us":30830,"wal-append.run_wall_time_us":1483400}
I20260709 13:13:24.466302  1003 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.430320  2906 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:24.471009   926 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:24.407019  2680 consensus_meta.cc:287] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.577s	user 0.019s	sys 0.000s
W20260709 13:13:24.294127  2951 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:13:24.282681  2899 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:24.432556  2931 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:24.464998  2686 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:24.351502  2829 consensus_meta.cc:287] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.003s
W20260709 13:13:24.474617   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.483489  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.473764   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.234673  2878 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.363593  2893 raft_consensus.cc:2272] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:24.486490   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:24.487334   925 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:24.488965  2906 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.457444  2698 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.470628  2666 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.500478   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 164) took 1029 ms. Trace:
I20260709 13:13:24.538309   846 rpcz_store.cc:276] 0709 13:13:23.471516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.482580 (+ 11064us) service_pool.cc:224] Handling call
0709 13:13:24.500291 (+1017711us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54004097d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:24.500416 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067500800}
I20260709 13:13:24.501441  1597 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:24.465068  2743 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:24.470418  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:24.428885  2538 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.509116  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:24.514683   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1375) took 3055 ms. Trace:
I20260709 13:13:24.235460  2924 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:24.513890  1003 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:24.482254  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:24.629907   923 rpcz_store.cc:276] 0709 13:13:21.458972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.722692 (+1263720us) service_pool.cc:224] Handling call
0709 13:13:24.083565 (+1360873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.430555 (+346990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.514377 (+ 83822us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400444dd0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:24.514569 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3128679808}
I20260709 13:13:24.466298   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.524783   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.630769  1001 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:24.435200  2944 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:24.640064   997 raft_consensus.cc:2393] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:24.527501   998 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:24.477438   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:24.535954  2514 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:24.555755   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.626428  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:24.642213   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:24.524288  2954 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:24.630766  1600 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 2: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:24.523582   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.646970   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:24.647055   996 raft_consensus.cc:2376] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3. Re-sending same reply.
I20260709 13:13:24.456549  2673 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.632143   926 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:24.635802   925 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:13:24.542972  2856 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.007s
I20260709 13:13:24.545560   995 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:24.471944  2696 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.648043   923 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:24.485985  2680 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.659722   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:24.651015   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 154) took 2064 ms. Trace:
I20260709 13:13:24.661029   925 rpcz_store.cc:276] 0709 13:13:22.586444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.613627 (+1027183us) service_pool.cc:224] Handling call
0709 13:13:23.614035 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.092959 (+478924us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540011ae20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b411c0 00007f0d02b42231 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c
0709 13:13:24.650628 (+557669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.650951 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306190080}
I20260709 13:13:24.624711  2950 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:24.662307   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.651850   922 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:24.641369  2818 consensus_meta.cc:287] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.061s	user 0.014s	sys 0.007s
W20260709 13:13:24.658344   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 204517us
W20260709 13:13:24.681267   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 203902us
I20260709 13:13:24.685449   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:24.686885   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.687999   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.543793  2863 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:24.656653  2941 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 13:13:24.694257   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 530017us
I20260709 13:13:24.697980   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:24.699385   818 leader_election.cc:336] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.581s (SENT)
I20260709 13:13:24.680555  2906 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:24.707722   921 raft_consensus.cc:2368] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4.
I20260709 13:13:24.683279  2945 raft_consensus.cc:2764] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:24.640769  2538 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:24.711385   818 leader_election.cc:336] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.587s (SENT)
W20260709 13:13:24.709939   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 248264us
I20260709 13:13:24.713229   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:24.686517  2818 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:24.715838   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.716637   923 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:24.718255   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 158) took 1671 ms. Trace:
I20260709 13:13:24.718655   853 rpcz_store.cc:276] 0709 13:13:23.046951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.789804 (+742853us) service_pool.cc:224] Handling call
0709 13:13:23.790114 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.790481 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.790517 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.791006 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.870438 (+ 79432us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:23.870496 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.870608 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.718019 (+847411us) raft_consensus.cc:1685] finished
0709 13:13:24.718043 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.718197 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3052,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":78978,"threads_started":1,"wal-append.queue_time_us":499045}
I20260709 13:13:24.700928  1595 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.724998   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.725704   926 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:24.725821   853 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:24.726454   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1389) took 1468 ms. Trace:
I20260709 13:13:24.726761   853 rpcz_store.cc:276] 0709 13:13:23.258404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.725384 (+1466980us) service_pool.cc:224] Handling call
0709 13:13:24.725661 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.726192 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.726396 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.707713  2743 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:24.729638  1002 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:24.731129  2900 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.662900  2856 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.735247   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 161) took 1405 ms. Trace:
I20260709 13:13:24.735656   853 rpcz_store.cc:276] 0709 13:13:23.330193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.734668 (+1404475us) service_pool.cc:224] Handling call
0709 13:13:24.734892 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.735009 (+   117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.735022 (+    13us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.735049 (+    27us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:24.735066 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.735097 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.735167 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.737015   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 163) took 1289 ms. Trace:
I20260709 13:13:24.737309   853 rpcz_store.cc:276] 0709 13:13:23.448078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.736402 (+1288324us) service_pool.cc:224] Handling call
0709 13:13:24.736675 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.736818 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.736832 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.736858 (+    26us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:24.736876 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.736907 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.736977 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.739619  2696 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:24.731917  2900 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:24.760468   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:24.754328   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1485) took 1824 ms. Trace:
I20260709 13:13:24.763214   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.763845   926 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:24.695874  2950 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.764869   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:24.765553  2919 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:24.749647  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:24.766358  2950 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.692601  2931 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.750481  2666 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.752081   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1486) took 1817 ms. Trace:
I20260709 13:13:24.775718  2863 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:24.776381   847 rpcz_store.cc:276] 0709 13:13:22.934675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.677058 (+742383us) service_pool.cc:224] Handling call
0709 13:13:24.503333 (+826275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.503525 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.503554 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.503618 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:24.503660 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.503722 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.751914 (+248192us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54004097d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:24.752025 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796012928}
I20260709 13:13:24.778002   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.767432  2931 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:24.782799   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.761937   849 rpcz_store.cc:276] 0709 13:13:22.930381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.675925 (+745544us) service_pool.cc:224] Handling call
0709 13:13:23.690810 (+ 14885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.092985 (+402175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.093059 (+    74us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.093197 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:24.093384 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.754112 (+660728us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54001ccfd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:24.754155 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.754276 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605804928}
I20260709 13:13:24.798390   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.847159   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.698598  2899 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:24.850049   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.717309  2924 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.748158  2958 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 13:13:24.855125   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:24.779423   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.708554  2944 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.862618  2918 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.774258  2856 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:24.869575   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.914947   927 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:24.915431  2680 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.916139  2698 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.917569   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.918926   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:24.921079   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.923411  2829 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:24.924235  2829 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.924938   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.926172   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.927575  2829 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:24.877604   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.929260   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:24.930482   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:24.931150   849 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.759078  2900 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:24.933233  2900 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:24.934684  2900 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.885989  2958 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:24.886958   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.943323   929 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:24.890360  1566 connection.cc:256] RPC call timeout handler was delayed by -1.1866848790000404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:13:24.895457   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1481) took 1966 ms. Trace:
I20260709 13:13:24.895888  2965 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:13:24.896449  2893 ts_tablet_manager.cc:1916] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:13:24.898945  2924 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:24.901119   854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 157) took 1953 ms (client timeout 1714 ms). Trace:
I20260709 13:13:24.927345   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:24.902779   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47778.8us
I20260709 13:13:24.903429  2666 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:24.903784  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 262) took 1149 ms. Trace:
I20260709 13:13:24.909092   922 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:24.910609  2538 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:24.923575  2944 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:24.874495   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1487) took 1935 ms. Trace:
I20260709 13:13:24.963518   855 rpcz_store.cc:276] 0709 13:13:22.939649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.689159 (+749510us) service_pool.cc:224] Handling call
0709 13:13:24.873472 (+1184313us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:24.873782 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.874012 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.874050 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.874106 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:24.874153 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.874225 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.874419 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3293568}
I20260709 13:13:24.884505  1592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 263) took 1129 ms. Trace:
I20260709 13:13:24.971989  1592 rpcz_store.cc:276] 0709 13:13:23.754676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.760250 (+  5574us) service_pool.cc:224] Handling call
0709 13:13:23.778844 (+ 18594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.837962 (+ 59118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.884124 (+1046162us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b540045d8d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:24.884418 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119232896}
I20260709 13:13:24.937323   894 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:24.973435   894 raft_consensus.cc:2686] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:13:24.938176  2673 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:24.971989   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.973460   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.979735   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.980556   923 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
W20260709 13:13:24.981302   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.981501   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.982357   923 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:24.945760   848 rpcz_store.cc:276] 0709 13:13:22.929242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.953692 (+ 24450us) service_pool.cc:224] Handling call
0709 13:13:24.894510 (+1940818us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:24.894780 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.895019 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.895066 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.895130 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:24.895174 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.895246 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.895382 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6231424}
I20260709 13:13:24.944762   928 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:24.952744   854 rpcz_store.cc:269] 0709 13:13:22.948235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.384336 (+436101us) service_pool.cc:224] Handling call
0709 13:13:24.900866 (+1516530us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b540023b254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:24.901035 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778083712}
I20260709 13:13:24.901971   853 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:24.951140  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:24.955736   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:24.944417  2958 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:24.993424   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.959808  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 206005us
I20260709 13:13:24.952478  2952 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:24.970774   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.976941   921 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:24.994530   926 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:24.992697   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:24.995563  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:24.996100   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.990643  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:24.997563   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:24.998019   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:24.999435   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:24.999790   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.983320   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.000850   923 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.002399   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:24.998631   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:25.003536   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:25.005224   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 293838us
I20260709 13:13:25.005537   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:25.006105   818 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.601s (SENT)
I20260709 13:13:25.006636   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.009773   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.010808  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.012194  2538 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.012689  2538 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.002166  2965 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:13:25.004982  2959 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:25.013793   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.014472   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:25.014680  2538 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.015301   847 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:25.015991   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.016700  2698 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:25.016922  2966 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:24.996929  2665 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:25.017081  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:25.017572  2665 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:25.017765  2967 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:25.019860   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.020731  2918 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.021011  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.021168  2918 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:25.021651  2918 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.022197  2935 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.023198  2917 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.019812  1564 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.023823  2666 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.004442   930 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:25.025832  2666 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.026219  2666 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:25.026598  2666 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.026626   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.013219  1001 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:25.018055  2665 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:24.952023  2964 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:13:25.045697  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.049342  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.050273   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:25.050843   995 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:25.054788   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.055449   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 370033us
I20260709 13:13:25.017582  2966 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.017144  2698 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.038017  2919 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.046620  2943 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:13:25.070258   849 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:25.011688  2952 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:24.962267  1595 rpcz_store.cc:276] 0709 13:13:23.754372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.756422 (+  2050us) service_pool.cc:224] Handling call
0709 13:13:23.772228 (+ 15806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.742838 (+970610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.903491 (+160653us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b5400710f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:24.903720 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107544448}
I20260709 13:13:25.073632   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1398) took 1538 ms. Trace:
W20260709 13:13:25.084997   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.141695   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.142386   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.094532  2918 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.066987  2958 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:25.098887   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.104372   750 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 224) took 12876 ms (client timeout 15000 ms). Trace:
I20260709 13:13:24.961205 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:25.108327   753 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.114938   926 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:25.120428  2966 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.154428  2969 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.155754  2958 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.157367   923 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:25.064981   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.140125   929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1371) took 3680 ms (client timeout 2222 ms). Trace:
I20260709 13:13:25.140635   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:25.163280   929 rpcz_store.cc:269] 0709 13:13:21.450018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.553732 (+1103714us) service_pool.cc:224] Handling call
0709 13:13:25.130433 (+2576701us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b5400169ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:25.130604 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5351289600}
I20260709 13:13:25.141392   853 rpcz_store.cc:276] 0709 13:13:23.535497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.738030 (+1202533us) service_pool.cc:224] Handling call
0709 13:13:24.901593 (+163563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.993821 (+ 92228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.073362 (+ 79541us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540023b250. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:25.073547 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375450880}
I20260709 13:13:25.177609   851 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:25.179399   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1495) took 2049 ms. Trace:
I20260709 13:13:25.180368   852 rpcz_store.cc:276] 0709 13:13:23.129583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.089142 (+959559us) service_pool.cc:224] Handling call
0709 13:13:24.089525 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.089762 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:24.431179 (+341417us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.732474 (+301295us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.732804 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.858406 (+125602us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b1800400ea0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa4f455 00007f0d02b37060 00007f0d02b49716 00007f0d02b5a287 00007f0d02b5a208
0709 13:13:24.875254 (+ 16848us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b1800400ea0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:24.932707 (+ 57453us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b1800400ea0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:24.970159 (+ 37452us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b1800400ea0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa52522 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:25.177000 (+206841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.177079 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:25.177156 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.177249 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.179335 (+  2086us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56367,"prepare.queue_time_us":634604,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":129,"raft.queue_time_us":187463,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":53218,"spinlock_wait_cycles":170471168,"thread_start_us":952237,"threads_started":3,"wal-append.queue_time_us":356608}
W20260709 13:13:25.189550   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.192865   896 connection.cc:256] RPC call timeout handler was delayed by -2.0674439410001924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:13:25.193347   896 connection.cc:256] RPC call timeout handler was delayed by -2.0197439410001152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:13:25.213328   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.144526   750 rpcz_store.cc:269] 0709 13:13:12.228410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.256157 (+ 27747us) service_pool.cc:224] Handling call
0709 13:13:25.104277 (+12848120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:22.295578 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:22.312720 (+ 17142us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:22.312785 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:13:22.312806 (+    21us) tablet.cc:662] Decoding operations
0709 13:13:22.313297 (+   491us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:22.313372 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:22.313406 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:13:22.313837 (+   431us) tablet.cc:701] Row locks acquired
0709 13:13:22.313881 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:13:22.314068 (+   187us) write_op.cc:270] Start()
0709 13:13:22.314225 (+   157us) write_op.cc:276] Timestamp: P: 1783602802314000 usec, L: 0
0709 13:13:22.314244 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:13:22.593126 (+278882us) log.cc:844] Serialized 923 byte log entry
0709 13:13:23.483740 (+890614us) op_driver.cc:464] REPLICATION: finished
0709 13:13:23.692812 (+209072us) write_op.cc:301] APPLY: starting
0709 13:13:23.695520 (+  2708us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:23.699211 (+  3691us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:23.703030 (+  3819us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:23.703556 (+   526us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:23.731852 (+ 28296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:13:23.731911 (+    59us) write_op.cc:312] APPLY: finished
0709 13:13:23.732201 (+   290us) log.cc:844] Serialized 29 byte log entry
0709 13:13:23.732433 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:23.732537 (+   104us) write_op.cc:454] Released schema lock
0709 13:13:23.732605 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:23.733063 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:23.733099 (+    36us) write_op.cc:454] Released schema lock
0709 13:13:24.772600 (+1039501us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3c32a 00007f0d02ac1f14 00007f0d02ac1ec3 00007f0d02ac1e38 00007f0d02ac1df3 00007f0d02ac10dc 00007f0d02aeae16 00007f0d02ad2cbb 00007f0d02abd39d 00007f0d02ac0d75
Metrics: {"mutex_wait_us":987857,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":205825,"apply.run_cpu_time_us":2967,"apply.run_wall_time_us":40246,"num_ops":2,"prepare.queue_time_us":865611,"prepare.run_cpu_time_us":50509,"prepare.run_wall_time_us":613306,"raft.queue_time_us":276445,"raft.run_cpu_time_us":78128,"raft.run_wall_time_us":1331082,"replication_time_us":1146117,"spinlock_wait_cycles":7091584,"thread_start_us":943122,"threads_started":4,"wal-append.queue_time_us":455724,"wal-append.run_cpu_time_us":46322,"wal-append.run_wall_time_us":1624104}]]}
W20260709 13:13:25.193733   896 connection.cc:256] RPC call timeout handler was delayed by -1.9909439410000687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 13:13:25.235286   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1388) took 1998 ms. Trace:
W20260709 13:13:25.280416   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 223945us
W20260709 13:13:25.238310   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 298042us
I20260709 13:13:25.286684   851 rpcz_store.cc:276] 0709 13:13:23.237205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.180716 (+943511us) service_pool.cc:224] Handling call
0709 13:13:25.177392 (+996676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.191169 (+ 13777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.235084 (+ 43915us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54000fee50. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:25.235231 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167269248}
W20260709 13:13:25.280426  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 232898us
W20260709 13:13:25.291435   896 connection.cc:256] RPC call timeout handler was delayed by -1.9234439410001869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 13:13:25.291988   896 connection.cc:256] RPC call timeout handler was delayed by -1.7758439410001756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 13:13:25.292443   896 connection.cc:256] RPC call timeout handler was delayed by -1.7704439410001669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:13:25.292502  1564 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.170022  2929 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:25.292975   896 connection.cc:256] RPC call timeout handler was delayed by -1.7560439410001436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 13:13:25.293464   896 connection.cc:256] RPC call timeout handler was delayed by -1.6939439410000432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 13:13:25.293826   896 connection.cc:256] RPC call timeout handler was delayed by -1.68584394100003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
W20260709 13:13:25.294188   896 connection.cc:256] RPC call timeout handler was delayed by -1.6462439410001934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:13:25.294540   896 connection.cc:256] RPC call timeout handler was delayed by -1.5859439410000959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 13:13:25.294890   896 connection.cc:256] RPC call timeout handler was delayed by -1.5778439410000829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 13:13:25.295253   896 connection.cc:256] RPC call timeout handler was delayed by -1.52204394100022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 13:13:25.262172   995 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
W20260709 13:13:25.305757   896 connection.cc:256] RPC call timeout handler was delayed by -1.4752439410001443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
W20260709 13:13:25.306324   896 connection.cc:256] RPC call timeout handler was delayed by -1.4617439410001225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:13:25.309468   896 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:25.308845   753 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.192559  2900 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:25.191416  2696 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.318449   895 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.330180   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.330681   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:25.233633  2538 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.238117  2944 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:25.322732  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 326892us
I20260709 13:13:25.339289   853 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:25.341917  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.311553  2919 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:25.346681   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.319670  2931 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.317057  2900 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.349068  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.329595  2967 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:25.330195  2943 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.357950   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.348433   753 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.337885  2961 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:25.368460  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.334372  2686 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.369868   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:25.375335   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.340346  2929 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:25.354305  2968 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.340356  2514 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:25.349442  2901 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:25.397174   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 391060us
W20260709 13:13:25.411022   818 leader_election.cc:336] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.602s (SENT)
I20260709 13:13:25.358989  2919 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:25.356494  2967 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.351080  2959 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.340173  2680 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:25.319553  2698 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.361739  2899 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:25.409202   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.474697   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.347343  2538 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.353292  2941 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:13:25.395155  2931 raft_consensus.cc:2764] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:25.418370   896 leader_election.cc:400] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:25.477581   896 leader_election.cc:403] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:25.422412  2680 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:25.430527  2959 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.370050  2943 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:25.428227  2971 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:13:25.435021  2929 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.394150  2964 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:25.378244  2961 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.440659  2665 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.417054  2666 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.414250  2944 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:25.397657  2984 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:13:25.403153  2915 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:13:25.427320  2973 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:13:25.457382   753 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 3, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:25.477999  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 129005us
I20260709 13:13:25.479843  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.436719  2952 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.458873  2901 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:25.487294   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 107852us
I20260709 13:13:25.410084  2900 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:25.488627   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 112268us
W20260709 13:13:25.490501   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 147317us
I20260709 13:13:25.494537   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:25.531657  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51903us
I20260709 13:13:25.531946  2952 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.532557   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:25.496942   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 84987.6us
I20260709 13:13:25.533434   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.534653   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:25.536693   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.497516   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.486863  2680 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:25.537489   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.491176  2686 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:25.542750   852 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:25.495347  2514 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:25.543298  2514 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:25.544142  2514 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.521123   995 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:25.496490  2698 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.516127  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.526379   753 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.486172  2899 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.512066  2944 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:25.570019  2982 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:13:25.570515   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 160) took 2328 ms. Trace:
I20260709 13:13:25.570662  2944 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.570945   850 rpcz_store.cc:276] 0709 13:13:23.242034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.526716 (+1284682us) service_pool.cc:224] Handling call
0709 13:13:24.626484 (+ 99768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.700609 (+ 74125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.700653 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.710438 (+  9785us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.189053 (+478615us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:25.189114 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.189243 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.570280 (+381037us) raft_consensus.cc:1685] finished
0709 13:13:25.570300 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.570434 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112414,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":503488,"threads_started":2,"wal-append.queue_time_us":586454}
I20260709 13:13:25.572019  2961 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:25.572932  2978 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:13:25.572876   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.574237   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1496) took 2329 ms. Trace:
W20260709 13:13:25.574913   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.575464   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.574717   846 rpcz_store.cc:276] 0709 13:13:23.244470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.539560 (+1295090us) service_pool.cc:224] Handling call
0709 13:13:24.540367 (+   807us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.856605 (+316238us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:13:24.856653 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.858039 (+  1386us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.233811 (+375772us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:25.233846 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.266977 (+ 33131us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400177fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:25.267047 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.573906 (+306859us) raft_consensus.cc:1685] finished
0709 13:13:25.573924 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.574168 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84112,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"spinlock_wait_cycles":263437056,"thread_start_us":483596,"threads_started":2,"wal-append.queue_time_us":397105}
I20260709 13:13:25.576876   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.577518   846 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:25.577376   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:25.578291   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.578414   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.578980   846 raft_consensus.cc:2368] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:25.553954  2952 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:25.580047  2900 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.580789   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.581807   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.582289   923 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.583906  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:25.584534  1600 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:25.584621   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.585328   921 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:25.586233   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.587834   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.551265   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:25.588486  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:25.567595  2666 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.567708   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.530778  1595 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:25.569226  2538 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.569567  2899 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:25.571868  2944 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:25.574178   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:25.576062   818 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.620s (SENT)
I20260709 13:13:25.560897   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:25.588795   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.615187   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.590284   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:25.617066   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.587311   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.563378   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.562829   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.588608   930 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 2: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:25.579495   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.594512   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.606784   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.613767   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.621039   922 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.613094   753 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.624491   753 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:25.574815  2989 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:13:25.489960  2967 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.631875  2905 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.636162  2905 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.637957   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.638384  2905 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:25.638581   926 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.639717  2538 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.642655  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.584981  2665 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.644259   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.644378   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:25.626850   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.645551   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.646071   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.618616   927 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:25.647128   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.618616   921 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:25.626259  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:25.628525   923 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:25.618850   928 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:25.631394   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:25.632078   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.648952   930 raft_consensus.cc:2376] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4. Re-sending same reply.
I20260709 13:13:25.632555  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:25.630970  2954 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:13:25.634768  2983 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:13:25.570544  1592 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:25.635251   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.649308  1002 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:25.494532  2979 consensus_queue.cc:1059] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:13:25.492691  2919 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.587826  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:25.692610   848 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:25.693210   968 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.650892   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 74851.2us
I20260709 13:13:25.653088   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:25.653502   846 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:25.694161   818 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.639s (SENT)
I20260709 13:13:25.654728   896 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.694579   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.656574   928 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:25.614979  2961 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:25.695851   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:25.519922  2988 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
W20260709 13:13:25.698067  2993 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:25.700220  2979 raft_consensus.cc:3055] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:25.700650  2979 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.701529  2979 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:25.702843  2979 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.695851   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.666584   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:25.705516   928 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.692008   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:25.706395   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51742us
I20260709 13:13:25.659978   922 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:25.654982   926 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:25.652076   753 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 4, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.707234   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.711054  1004 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:25.627743  2991 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.606729  2680 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.657809  2968 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.644193  2964 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.627523  2931 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.726444   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.752962   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.711714  2538 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.754406   927 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:25.754645   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:25.713668  1596 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.759020   895 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
W20260709 13:13:25.759279   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.764243   895 leader_election.cc:403] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:25.759871   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52727.6us
I20260709 13:13:25.755056   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.710436  2905 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.765390   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.773574   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:25.751422  1597 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:25.749055  2958 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.777475   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.782212   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:25.813390   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48041.8us
I20260709 13:13:25.821573   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.766520  2862 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.768718  2686 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:25.823190   896 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.782332  2929 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.759398   853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:25.834563   853 raft_consensus.cc:606] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:25.834911   853 raft_consensus.cc:614] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:25.832556   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 55201.7us
I20260709 13:13:25.782475  2905 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:25.836228   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.837884   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.767871  2986 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:13:25.842964  2986 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:25.843468  2986 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:25.845868   895 leader_election.cc:400] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 2. Current term is 3.
I20260709 13:13:25.863507   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.909165   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:25.910398   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.815510  2914 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.826651  2991 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.845810   896 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.934608   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 88853.4us
I20260709 13:13:25.936299   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.886669   928 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:25.938288   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:25.939175   928 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:25.918450   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:25.924156   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 78399.9us
I20260709 13:13:25.942639   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.944087  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:25.926460   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 152977us
I20260709 13:13:25.945488   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.860811  3007 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.947575  3007 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.948609  3007 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.950760  3007 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:25.951894  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 240110us
W20260709 13:13:25.954413   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 251109us
W20260709 13:13:25.852092  2975 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:13:25.854123  3006 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:25.847268  2538 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.869464  2680 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.854597  3010 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.955032   925 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:25.845185  2905 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.962742  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:25.865064  2918 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.852288  2980 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:25.824560  2958 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:25.916314  2686 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.965950   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:25.915072  2943 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:25.868439  2965 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:25.845583  2893 ts_tablet_manager.cc:1929] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:13:25.960486  2514 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.959722  2931 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.980064   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:25.984403   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48223.7us
I20260709 13:13:25.986068   896 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:25.969735  2914 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:25.963402  2915 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:13:25.972147  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:26.009209  2893 log.cc:1199] T 90a3708334cc4bd59c73d494d2747d54 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/90a3708334cc4bd59c73d494d2747d54
I20260709 13:13:25.995272  2899 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:26.000553  3024 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:13:26.016188  3005 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.016357  2994 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:13:26.041610   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55618.7us
I20260709 13:13:26.019838  1596 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:26.043191   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:26.043159   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 100642us
I20260709 13:13:25.976217  2965 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:26.026677  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:25.984830  2968 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.982409  2959 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:13:26.039937  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:25.846146  2986 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.044724   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.066111   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.044773   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:26.049744  1002 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:26.068670   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:26.001950  2914 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:26.032558  2899 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:26.041122  3005 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.069087   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1501) took 1323 ms. Trace:
I20260709 13:13:26.114243   847 rpcz_store.cc:276] 0709 13:13:24.745776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.082836 (+337060us) service_pool.cc:224] Handling call
0709 13:13:25.083205 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.083521 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:25.192268 (+108747us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.997484 (+805216us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.047177 (+ 49693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.056055 (+  8878us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:26.056123 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.056224 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.064244 (+  8020us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37522,"prepare.queue_time_us":114299,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":5475,"raft.queue_time_us":142810,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":13414,"spinlock_wait_cycles":13092864,"thread_start_us":615018,"threads_started":3,"wal-append.queue_time_us":239429}
I20260709 13:13:26.041844  3012 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:26.118352  3012 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:26.118854  3012 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:26.096766  2686 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:26.098388   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.062443  2538 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:26.064364  2961 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 13:13:26.104509  2900 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
I20260709 13:13:26.106591  2991 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:26.056547  3023 log.cc:927] Time spent T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:13:26.082643  2914 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:26.111883  2893 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42376 (request call id 74) took 4716 ms. Trace:
I20260709 13:13:26.110108  2899 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.155651  2991 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.156973  1593 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:26.158527  2991 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.129668   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.133939   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.146111  2944 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:26.149024  2538 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:26.152661   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.153813  3026 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.124962  3005 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.180395  2965 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.182331  2965 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:26.146082  2893 rpcz_store.cc:276] 0709 13:13:21.395565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.395859 (+   294us) service_pool.cc:224] Handling call
0709 13:13:22.387192 (+991333us) ts_tablet_manager.cc:1151] Deleting tablet 90a3708334cc4bd59c73d494d2747d54
0709 13:13:22.387318 (+   126us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:13:25.350793 (+2963475us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:25.829022 (+478229us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:26.111757 (+282735us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1353010,"tablet-delete.queue_time_us":990290,"thread_start_us":257200,"threads_started":1}
I20260709 13:13:26.186094   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 90a3708334cc4bd59c73d494d2747d54 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:13:26.145404   855 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:26.160962   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.174099  2944 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.174942  2999 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.175912  2538 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.177985   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.102034  3010 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:26.200829  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:26.191473  2964 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:26.206387  2964 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:26.145797  2929 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:26.208058  1593 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:26.213238  2964 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.210757   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.218034   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.218755   930 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:26.200923  2979 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.200553  3026 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:26.222496   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.233031   930 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:26.234308   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.201159  2999 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.224609   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:26.232411   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:26.224213   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 90391.5us
I20260709 13:13:26.238497   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.236997   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:26.240718   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 260773us
W20260709 13:13:26.241739   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 180383us
I20260709 13:13:26.241551   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.208652  2686 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:26.243355   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.213939  2905 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.248724   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.232280  2979 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.235410  3010 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.239918  2999 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.256320   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.245267   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.256304   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.260716   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.262221   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.265136   926 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:26.269361   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.246024  2966 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:26.308065   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46064.6us
I20260709 13:13:26.295264   927 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:26.252633  3026 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:26.319021   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 191532us
I20260709 13:13:26.254918  2979 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:26.319978   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 74828.6us
I20260709 13:13:26.332650   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:26.332075   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 62884.2us
I20260709 13:13:26.331005   930 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:13:26.334177   930 pending_rounds.cc:85] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 1
I20260709 13:13:26.334561   930 pending_rounds.cc:107] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:13:26.334324   895 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.334849   930 raft_consensus.cc:2889] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:26.270397  2900 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:26.267383  2698 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:26.266194  2964 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.336548   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.338477   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:26.257468  2958 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.318508  2966 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.321219  3024 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.348905   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:26.336413  2906 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:26.352833  2966 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.342833  2698 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.353088  3007 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:26.270284  2915 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:13:26.357270  3028 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:26.357445   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:26.363822  2666 raft_consensus.cc:2764] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:26.269706  2686 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.364486  2997 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:26.401052  2997 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:26.401181  3033 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:26.401988  2997 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.342681  2900 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.402593  3016 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.266906  3010 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.344098  2959 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.402988  2900 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:26.406359   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.407126   921 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:26.352584  3024 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.364650  2958 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:26.319283  2914 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.351171  2680 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:26.399246  3027 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.400352   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:26.382077   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 117149us
W20260709 13:13:26.384014   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 47547.1us
I20260709 13:13:26.369767  2698 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.411885   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.470669   928 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:26.411988  3024 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.461736  2980 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.471411  2906 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:26.483084   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 142580us
I20260709 13:13:26.447034  3017 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:26.530771   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.522294  2997 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.521334  3016 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.523264  2959 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:26.540596   968 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.4:33689 timed out after 1.857s (SENT) [suppressed 68 similar messages]
I20260709 13:13:26.540865   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:26.543937   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67299.7us
I20260709 13:13:26.546864  2915 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:26.547072   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1402) took 1017 ms. Trace:
I20260709 13:13:26.550275   855 rpcz_store.cc:276] 0709 13:13:25.529790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.532320 (+  2530us) service_pool.cc:224] Handling call
0709 13:13:26.546929 (+1014609us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b540040a1d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:26.547023 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407745792}
I20260709 13:13:26.487058  2980 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.547462  2915 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:26.557180  3037 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:13:26.557677  2906 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.562215   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:26.561326  3032 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:13:26.568248   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.575227   930 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:26.575147  2905 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.551828  2959 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.606035  2959 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.606218  2901 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.606712  2901 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.580372  3044 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:26.593461  3012 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.608140  3044 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.608811  2901 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:26.609452  3012 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:26.610105   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47973.6us
I20260709 13:13:26.610836   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:26.596082  3010 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.612571   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.597723  2698 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.574466  3005 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.611959  3034 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:26.614343   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.614974   855 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:26.578728  2979 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.548512  3016 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:26.623347  3041 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:13:26.623679  3012 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:26.624017  3045 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.625232  3016 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.609105  3044 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.631035  1594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 371) took 1050 ms. Trace:
I20260709 13:13:26.632328  2899 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.634897  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:26.635900  1599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 378) took 1053 ms. Trace:
I20260709 13:13:26.636343  1599 rpcz_store.cc:276] 0709 13:13:25.582434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.610057 (+ 27623us) service_pool.cc:224] Handling call
0709 13:13:26.634760 (+1024703us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:26.635789 (+  1029us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40685440}
I20260709 13:13:26.638082  2961 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:26.639097  2680 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.634454  2899 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:26.619690  3024 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.639549  2961 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:26.639816  2680 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.650380  2915 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.652024   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1386) took 1091 ms. Trace:
I20260709 13:13:26.641065  3012 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.652526   925 rpcz_store.cc:276] 0709 13:13:25.560379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.560610 (+   231us) service_pool.cc:224] Handling call
0709 13:13:26.651798 (+1091188us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400174154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:26.651948 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627527936}
I20260709 13:13:26.638083   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1490) took 1280 ms. Trace:
I20260709 13:13:26.654656   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.654407   995 rpcz_store.cc:276] 0709 13:13:25.357451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.357659 (+   208us) service_pool.cc:224] Handling call
0709 13:13:26.637811 (+1280152us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001dc254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:26.637956 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373446528}
I20260709 13:13:26.655041   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.656590   921 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:26.655382   930 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:26.661252   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 112593us
I20260709 13:13:26.662499   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:26.663154   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:26.663815   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.665112   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:26.665429   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.665680   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.666447   848 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:26.669152   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:26.668076   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.671139  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:26.671759   846 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:26.581449  2999 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.675936   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.677443   849 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:26.645138  3024 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.653246  3012 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.642274  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 414) took 1351 ms. Trace:
I20260709 13:13:26.643123  3005 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.675557   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:26.664799  2914 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:26.664278  2965 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:26.664558  2686 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.672679  2966 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.665194  2900 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.688374  3010 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.631464  1594 rpcz_store.cc:276] 0709 13:13:25.580402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.613168 (+ 32766us) service_pool.cc:224] Handling call
0709 13:13:26.630270 (+1017102us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:26.630467 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.630680 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.630712 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.630754 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:26.630791 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.630856 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.630975 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49165056}
I20260709 13:13:26.689916  2924 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:26.673280  3016 raft_consensus.cc:2764] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:26.699268   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:26.675905  3028 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:26.702040   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 122793us
I20260709 13:13:26.696817   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.624425  3043 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:13:26.714658   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79826.5us
I20260709 13:13:26.718519  3043 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:26.718906  3043 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.719739  3043 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.721105  3043 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.724277  2960 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:26.724869  2960 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.726380  3054 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.726914  3054 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.727311  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:26.728156  1598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 375) took 1146 ms. Trace:
I20260709 13:13:26.728484  1598 rpcz_store.cc:276] 0709 13:13:25.581596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.609978 (+ 28382us) service_pool.cc:224] Handling call
0709 13:13:26.727137 (+1117159us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:26.728073 (+   936us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39418112}
I20260709 13:13:26.729135  3054 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:26.729216  2966 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.729720   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 221) took 1089 ms. Trace:
I20260709 13:13:26.730075   999 rpcz_store.cc:276] 0709 13:13:25.640214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.642026 (+  1812us) service_pool.cc:224] Handling call
0709 13:13:26.639913 (+997887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.640137 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.640175 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.640243 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.640287 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.640345 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.729561 (+ 89216us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54001dc250. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:26.729670 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294068096}
I20260709 13:13:26.731040  2999 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.732069  2980 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.705155  3010 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.732692  3010 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.700964  2965 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.735487   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.737159  2965 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.737534  2680 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.738951   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.709812  3028 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:26.739727  3028 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.739910   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:26.740546   921 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:26.706436  2924 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:26.684147  1595 rpcz_store.cc:276] 0709 13:13:25.291206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.299546 (+  8340us) service_pool.cc:224] Handling call
0709 13:13:25.354850 (+ 55304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.632964 (+278114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.642042 (+1009078us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540045cc54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:26.642215 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364709248}
I20260709 13:13:26.742199  2698 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.741830  2924 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.742892  2924 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.743396  2979 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.712435  2686 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:26.745292  2686 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.745641  2686 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.746130   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.747991   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1414) took 1029 ms. Trace:
I20260709 13:13:26.748428   855 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:26.748528   926 rpcz_store.cc:276] 0709 13:13:25.718918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.918102 (+199184us) service_pool.cc:224] Handling call
0709 13:13:26.747760 (+829658us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54003cbc54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:26.747916 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644201216}
I20260709 13:13:26.749750   925 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:26.748888   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.758522   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:26.759491   852 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:26.762245   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:26.762777   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.763846   921 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:26.768281   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.775368   852 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:26.753578   847 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:26.758980  2698 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.762002  2828 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:26.784518   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:26.785723   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:26.786243   852 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:26.787645   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:26.792270  2828 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.780172  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:26.822476   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:26.824198   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:26.829756   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.786649  2698 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.844514   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:26.823580  1594 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:26.819108  2680 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:26.836063  3016 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:26.842026  3044 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:26.843272  2862 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:13:26.866575  3016 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:26.888625  3056 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:13:26.879791  2900 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.879760  3044 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.899401  1600 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:26.899427  3055 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:26.900285  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 379) took 1317 ms. Trace:
I20260709 13:13:26.900642  1600 rpcz_store.cc:276] 0709 13:13:25.582727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.650036 (+ 67309us) service_pool.cc:224] Handling call
0709 13:13:26.899007 (+1248971us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:26.899226 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.900027 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.900227 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5484416}
I20260709 13:13:26.903908   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.904273   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:26.904698   995 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.905220  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:26.908083  2856 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:26.908507  2856 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:26.909377  3010 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.909442  2856 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.910837  2856 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.911492  2944 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.912562  3028 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.913396  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:26.913549   928 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:26.913946  1600 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.914985  2686 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.916381   818 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:26.919756   998 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:26.920228  3010 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.920585   999 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:26.920727  2698 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.921744  1600 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:26.925530  3012 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:26.926100   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 173) took 1432 ms. Trace:
I20260709 13:13:26.884680  2828 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:26.866796  2680 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:26.930183   995 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:26.926673  2862 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:26.886191  3016 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:26.937249   854 rpcz_store.cc:276] 0709 13:13:25.493784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.493877 (+    93us) service_pool.cc:224] Handling call
0709 13:13:25.494150 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.462314 (+968164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.462341 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.462379 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.462407 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.925858 (+463451us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54002b7d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:26.925905 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.926035 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2226038528}
I20260709 13:13:26.903591  3044 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:26.939697  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:26.932340  2919 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:26.976619   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 69081.8us
I20260709 13:13:26.977401  1600 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:26.978469  1598 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:26.981033   968 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:26.989887  2863 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:26.995219  2680 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:26.996415  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:27.001435  1029 heartbeater.cc:646] Failed to heartbeat to 127.31.120.62:45937 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.31.120.62:45937 timed out after 15.000s (SENT)
I20260709 13:13:27.004094   852 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:27.005254  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:27.006700  2686 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:26.979936  3010 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.009148  3010 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.010108   851 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:27.010511   851 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:27.011273   851 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.012272   851 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.988544  3053 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:13:27.013293  3016 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.014070  2953 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:27.013559   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:27.017033   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:27.018154   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.019140  2686 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:27.023134   894 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.011385  3010 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.027180   928 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:27.028110   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.033351   851 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:13:27.029067   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.034314   925 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:27.027288  1566 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:27.039304  2862 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:27.040444  2905 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.040944  2905 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.044714   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.039546  2863 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.046854  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.051635   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:27.047955   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.049786   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.049850  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.016310  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:27.050843   851 pending_rounds.cc:85] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 2
I20260709 13:13:27.056565   851 pending_rounds.cc:107] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:27.053458   996 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:27.053622   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.054070   925 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:27.057689   995 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:27.026051  3016 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.024605  2680 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:27.031975  2986 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:27.056939   851 raft_consensus.cc:2889] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:27.054636  3010 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:27.059489  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.038625  2944 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:27.044581  2914 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:27.062347  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:27.043198  3046 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.045650  2953 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:27.064836   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.067972   852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:27.050195  2924 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:27.080040   851 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:27.080237   852 raft_consensus.cc:606] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:27.086668   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:27.096376  3053 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:27.098096  3053 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.098573  3053 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:27.098122   849 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:27.107995  3028 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.062841  2919 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:13:27.070307  2686 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:13:27.084278  2944 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:27.111948  2863 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.112402  2944 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.096485  2953 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:27.116847   849 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:27.116633  2953 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.115470  2818 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:13:27.119203  2818 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:27.119652  2818 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.120482  2818 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.121603  2818 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:27.122052  3019 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.025s	user 0.003s	sys 0.000s
I20260709 13:13:27.123503   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1404) took 1557 ms. Trace:
I20260709 13:13:27.124140   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.124876   996 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:27.125330  2960 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.126636  2960 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:27.123883   923 rpcz_store.cc:276] 0709 13:13:25.565655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.658346 (+ 92691us) service_pool.cc:224] Handling call
0709 13:13:25.658768 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.659222 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.659271 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.659908 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.970788 (+310880us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:25.974420 (+  3632us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.974718 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.974826 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.121524 (+1146698us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:27.123224 (+  1700us) raft_consensus.cc:1685] finished
0709 13:13:27.123323 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.123445 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36371,"prepare.queue_time_us":232853,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":33713,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":27447,"spinlock_wait_cycles":36147072,"thread_start_us":310534,"threads_started":1,"wal-append.queue_time_us":379138}
I20260709 13:13:27.129081  2686 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.129654  2686 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.132119  2967 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.133903  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.074640  2914 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.140743   846 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:27.136617  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73459.8us
I20260709 13:13:27.137841  3053 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.072283  3033 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 13:13:27.143916  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64762.8us
I20260709 13:13:27.094542  2665 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:27.145094  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:27.144212  3044 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.161864  2917 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:27.162457  2917 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:27.168430  2968 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.168956  3033 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:27.169032  2968 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.140555  3043 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:27.170806  3043 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:27.171190  3043 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.168447  2914 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.172138  3043 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.172989  2914 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.173664  3043 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:27.173578  2917 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.175441  3028 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.169517  3033 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.083668  3046 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.163316  3044 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:27.194783   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44508.4us
I20260709 13:13:27.195017  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.146112  2919 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.156446  2686 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:27.195433  2751 raft_consensus.cc:2764] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:27.211099  1004 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
W20260709 13:13:27.212199  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 56043.7us
I20260709 13:13:27.171459  2968 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.221539  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:27.221517  3071 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:13:27.206228  2917 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.226191   894 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:27.196321  3028 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.198915  2906 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:27.204118  2919 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.246124  2919 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.248718  3046 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:27.249250   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.231106  3053 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:27.239990  3047 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.696s	user 0.004s	sys 0.000s
I20260709 13:13:27.244069  2901 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.253470  2901 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.256008  2901 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.226039  3033 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.257951  3028 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:27.229135  2968 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:27.259753   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.269052   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.278156  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:27.279062  1594 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:27.283800   995 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:27.284415   847 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:27.285892   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:13:27.287292   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.287562  2968 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.288122  2968 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.288534  2968 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:27.298436  3025 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.181s	user 0.004s	sys 0.000s
I20260709 13:13:27.299741   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1407) took 1676 ms. Trace:
I20260709 13:13:27.300215   922 rpcz_store.cc:276] 0709 13:13:25.623344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.708074 (+ 84730us) service_pool.cc:224] Handling call
0709 13:13:25.708527 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.781543 (+ 73016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.781574 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.822185 (+ 40611us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.981763 (+159578us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:25.981832 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.981923 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.297884 (+1315961us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:27.299343 (+  1459us) raft_consensus.cc:1685] finished
0709 13:13:27.299372 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.299662 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34095,"raft.queue_time_us":49440,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8315,"spinlock_wait_cycles":14649728,"thread_start_us":198064,"threads_started":2,"wal-append.queue_time_us":243593}
I20260709 13:13:27.260649  1595 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:27.313534   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.316418  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:27.317093  1594 raft_consensus.cc:2376] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:13:27.318190  2924 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.318598  1599 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:27.318722  2924 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.322039   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.322690   925 raft_consensus.cc:2376] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4. Re-sending same reply.
I20260709 13:13:27.323633   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.324247   927 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:27.324589   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.324260   926 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:27.325383   922 raft_consensus.cc:2376] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:13:27.326344  1564 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.327452  2673 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.327898  2673 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.328140   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:27.327569   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.328889  2960 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:27.329725  2856 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.329811  3028 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.330412  2856 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.331014   996 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:27.332675  2856 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:27.330504  3028 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.334379   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.334671  3028 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.335121   926 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:27.328293  2673 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.326408   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.335517  2829 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:27.336228   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.322238  2924 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:27.336386   847 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:27.336862   996 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:27.338063   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.341167  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:27.342341   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.343187   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.343773   855 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:27.344759  2905 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:27.346370   847 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:27.350360   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.352484  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:27.358357   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.358999   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.359683   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.360292   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.360980   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.361573   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:27.365424   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.365523   925 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:27.366005   928 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:27.366761   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:27.367676   855 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:27.367973   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.367437   925 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:27.371918   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.372517   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.373129   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.373732  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.373726   999 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:27.375182   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.379407   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:27.380713  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:27.397056   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46801.3us
I20260709 13:13:27.380059   924 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:27.400978   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.411227  1000 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:27.412374   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44762.9us
I20260709 13:13:27.416211   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:27.416633   930 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:27.416522  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.418140   925 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:27.421591   928 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:27.322923  3051 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:27.429054   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.430020   851 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
W20260709 13:13:27.410028  3074 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:13:27.435765   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.394170  3033 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:27.355163  2960 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.408828  3028 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.413111  3006 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.438400   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.412797  2901 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.419770  2944 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:27.491756   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.497047   855 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:27.497634   998 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
W20260709 13:13:27.499143   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60819.2us
I20260709 13:13:27.422564  3053 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:13:27.500382   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.442533  3006 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.507308   855 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:27.427963  3051 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.432353  2914 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:13:27.446110  3046 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:27.446352  2901 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:27.503326  3067 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:13:27.490425  2680 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.451112  2944 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.540951   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:27.547586   753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 221) took 15537 ms (client timeout 15000 ms). Trace:
I20260709 13:13:27.544641   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:27.521270  3006 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.533959   749 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:27.544080  2901 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.518971  3034 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:27.519265  2929 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:27.555562   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55255.6us
I20260709 13:13:27.569574   749 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:27.582999  3053 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:27.587913  2929 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.559890  2905 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:27.554703  3042 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:27.573213   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 7de58b0d5ad94d869fac3b0491143543 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:51112
I20260709 13:13:27.562619  2856 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:27.543916  2944 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:27.575191   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:27.592425   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.593324   749 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:27.550791  2680 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.559298  2960 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.602280   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.449453  3028 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.603219   749 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:27.629596   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 54506.5us
I20260709 13:13:27.637521  3042 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.638279   749 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 3 to 5, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:27.638626  3042 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.641583  3028 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:27.643230   924 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:27.643707   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 727299us
W20260709 13:13:27.644075  3079 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:13:27.644439   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:27.644380   749 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 2, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:27.645685  2698 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.645819   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.646101  2698 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.646528  2698 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.647075   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.646660   749 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 3 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:27.648178  3018 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:27.648610  3018 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.648998  3018 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:27.594791  3053 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:27.597117  2905 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:27.563138   753 rpcz_store.cc:269] 0709 13:13:11.993938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.994578 (+   640us) service_pool.cc:224] Handling call
0709 13:13:27.531173 (+15536595us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:25.947337 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:25.948139 (+   802us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:25.948181 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:13:25.948208 (+    27us) tablet.cc:662] Decoding operations
0709 13:13:25.949406 (+  1198us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:25.949499 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:25.949555 (+    56us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:13:25.950431 (+   876us) tablet.cc:701] Row locks acquired
0709 13:13:25.950456 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:13:25.950659 (+   203us) write_op.cc:270] Start()
0709 13:13:25.950805 (+   146us) write_op.cc:276] Timestamp: P: 1783602805950602 usec, L: 0
0709 13:13:25.950833 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:13:26.030105 (+ 79272us) log.cc:844] Serialized 3234 byte log entry
0709 13:13:26.700039 (+669934us) op_driver.cc:464] REPLICATION: finished
0709 13:13:26.825250 (+125211us) write_op.cc:301] APPLY: starting
0709 13:13:26.825368 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:26.825635 (+   267us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:26.825772 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:26.898829 (+ 73057us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:26.899165 (+   336us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:13:26.899222 (+    57us) write_op.cc:312] APPLY: finished
0709 13:13:26.899524 (+   302us) log.cc:844] Serialized 65 byte log entry
0709 13:13:26.933979 (+ 34455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:26.938396 (+  4417us) write_op.cc:454] Released schema lock
0709 13:13:26.978902 (+ 40506us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:27.033994 (+ 55092us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:27.038253 (+  4259us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":172602,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":124895,"apply.run_cpu_time_us":23223,"apply.run_wall_time_us":179821,"num_ops":8,"prepare.queue_time_us":125490,"prepare.run_cpu_time_us":59991,"prepare.run_wall_time_us":389058,"raft.queue_time_us":113678,"raft.run_cpu_time_us":21503,"raft.run_wall_time_us":117842,"replication_time_us":735727,"thread_start_us":463930,"threads_started":4,"wal-append.queue_time_us":319458}]]}
I20260709 13:13:27.634493  3010 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.635154  2818 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:27.635717  3046 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:27.707146   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.709821  2964 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.712509   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 178) took 1342 ms. Trace:
I20260709 13:13:27.601099  3051 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.683852  2958 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:27.716698   997 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:27.720323   853 rpcz_store.cc:276] 0709 13:13:26.370457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.370683 (+   226us) service_pool.cc:224] Handling call
0709 13:13:26.407963 (+ 37280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:27.710127 (+1302164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.710154 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:27.710206 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:27.710233 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.712262 (+  2029us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54000ff354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:27.712316 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.712454 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":535434,"raft.queue_time_us":535664,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":1762495360}
I20260709 13:13:27.682003  2915 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:13:27.707218  2818 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:27.731997   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 84979.5us
I20260709 13:13:27.733480   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.741689   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.743534   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.744213   847 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:27.743546   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.746915   854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:27.747591   854 raft_consensus.cc:606] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:27.747952   854 raft_consensus.cc:614] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:27.749960  2698 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.751951  1566 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:27.752226  2929 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:27.752969  2905 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.753986  3021 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:27.754720  2905 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.719524  2906 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.707247  3010 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:27.775367   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65742.3us
I20260709 13:13:27.737746  2818 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:27.783407   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.783892   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.784206   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.792956   850 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:27.741111  2915 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:27.738432  2965 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:27.761013  2998 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.789580   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.790629   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:27.791296   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57875.4us
I20260709 13:13:27.810310   851 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:27.810941   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:27.794195   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.725936  2964 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.795243   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.781004  2958 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:27.771943  2686 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:27.798055  3084 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 13:13:27.798754  3087 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:13:27.799312  1598 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:27.801096  3006 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.774571  3053 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.817364   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.818526   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.818897   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.835970   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:27.814031  2917 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.003s
I20260709 13:13:27.868021   847 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:27.848486  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 325146us
I20260709 13:13:27.772791  2906 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.876605   924 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:13:27.878535   924 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:27.879010   924 raft_consensus.cc:614] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:27.879495   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.879820   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:27.828151  2974 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:27.894037   930 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:13:27.897320   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.898284   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85951.9us
I20260709 13:13:27.901695  1598 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:27.876904  2959 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:27.907812  2673 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.003s
I20260709 13:13:27.908257  2673 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:27.911403   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:27.911983   847 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.908517  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:27.807493  3018 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.845284  2686 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.845481  2998 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.931840   849 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:27.933007   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:27.920284   850 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:27.922564   847 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:27.883445  2953 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:27.929556   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 327358us
I20260709 13:13:27.941001  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:27.949508   818 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.951738   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:27.953325   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:27.954188   847 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:27.960057   852 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:27.933667  3043 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
I20260709 13:13:27.923513  2698 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.984612  2979 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:27.985250  2979 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:27.985699  2979 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:27.989924  3018 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.991053  3010 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:27.991643  3010 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.992086  3010 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:27.994522   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:27.996964   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45895.8us
I20260709 13:13:27.999003  1001 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:27.964764  2986 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:27.933084  1595 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:27.960831  2900 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:27.992576  2906 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:28.022010   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 110124us
I20260709 13:13:27.931360  2974 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:13:27.993481  2924 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:27.964555  2953 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:28.055892   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 208009us
I20260709 13:13:28.027012  2986 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:28.061776   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 108526us
I20260709 13:13:28.064985   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:13:28.073815   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 69866.4us
I20260709 13:13:28.073140   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.082561   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.055826  2673 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:28.057348  2900 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:28.094188   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81428.4us
I20260709 13:13:28.091348  2967 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.096989  2967 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.097332  2967 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.100579  3093 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:28.087886  2698 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:28.104619  2698 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.105309  2959 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.105823  2959 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.107565  2900 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.108001  2964 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.112314  2964 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.075778  2986 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:28.065907  2924 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:28.084283  3018 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:28.112470  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:28.127254   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.128306   998 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:28.059520  3085 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 13:13:28.123334   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58506.5us
I20260709 13:13:28.135831   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:28.124490   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.124491  2686 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.144383   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:28.145241   997 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.151221   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.152797   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.108367  3097 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:13:28.126488  3006 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.123252  3018 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.122090  2899 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:28.165503  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:28.167245  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.117321  2964 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.155221  3105 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:28.143420  1595 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:28.148895  2686 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.125831  3104 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:13:28.182878   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 7de58b0d5ad94d869fac3b0491143543 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:13:28.184736   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:28.180960   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45236.6us
I20260709 13:13:28.162535  3101 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:28.168023  3043 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.197743  3118 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:28.198448  3118 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.192162  2686 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.198884  3118 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.199545  2964 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.200179  3119 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:28.200346  2686 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.200919  3119 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:28.201501  3119 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:28.188225  2944 consensus_meta.cc:287] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.007s
I20260709 13:13:28.175072  3018 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.146607  3106 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:13:28.186509  3105 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.187161  3107 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.205122  2974 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.214021   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:28.213936  2974 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.217262  2974 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.191918  3006 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:28.192408  3113 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:28.200989  3118 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.207789   997 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:28.191141   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:28.230505   998 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:28.206090   819 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:28.231192   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:28.232434   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.210352   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:28.201010  2686 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.193153  3101 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:28.187238  2968 consensus_meta.cc:287] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.898s	user 0.009s	sys 0.003s
W20260709 13:13:28.205519  2901 consensus_meta.cc:287] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.001s
I20260709 13:13:28.222834   849 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:28.194554  2998 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.227245   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:28.227963   928 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:28.228324   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:28.229058   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:28.214038  3100 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 13:13:28.203716   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51022.6us
I20260709 13:13:28.205610  3043 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.208771  2944 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.195463  2924 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.221982  3107 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.295023  3105 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:28.299062  2905 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.006s
W20260709 13:13:28.288492  3123 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:28.284732   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:28.350191   851 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
W20260709 13:13:28.279749   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 69475us
I20260709 13:13:28.353909   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.355782   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.221753  1597 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:28.278710   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.235140   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.358093  1004 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:28.358377   921 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
W20260709 13:13:28.358709   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 123630us
I20260709 13:13:28.359930   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.284092   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.360347  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.361243   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.354249   996 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
W20260709 13:13:28.287889  3114 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:13:28.384485   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.387451   924 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:28.367662   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.388616   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.389317   921 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:28.389991   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.391289   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.391888   926 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:28.392132  3117 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:28.392199  3116 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:28.386148  2901 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.395190  2901 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.393024  3116 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.213044  3119 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.397706  3116 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.386382  2998 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.398942  2998 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.399392  2998 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.400342   928 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:28.279145  3118 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:28.279551  2900 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.333364  3104 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:28.377027  3105 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:28.416354   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:28.362123  2974 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:28.383981  3101 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:28.292023  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:28.376688  3108 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:28.414515   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1568) took 1012 ms. Trace:
I20260709 13:13:28.442082   848 rpcz_store.cc:276] 0709 13:13:27.402127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.408152 (+  6025us) service_pool.cc:224] Handling call
0709 13:13:27.408559 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.715078 (+306519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.715107 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:27.722107 (+  7000us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.893205 (+171098us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:27.893261 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.897752 (+  4491us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:28.414218 (+516466us) raft_consensus.cc:1685] finished
0709 13:13:28.414245 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:28.414438 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":303794,"raft.queue_time_us":305985,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5918,"thread_start_us":148054,"threads_started":1,"wal-append.queue_time_us":282589}
I20260709 13:13:28.384167  2914 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:13:28.417378   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.332942  2953 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:28.226218  3006 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.388913  2979 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.229866  3113 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.454833  3006 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.455791  3113 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.383714  2968 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.456604  2680 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.458194  2968 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.459040   929 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:28.463035  2818 consensus_meta.cc:287] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.669s	user 0.007s	sys 0.001s
I20260709 13:13:28.446812  2944 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.447110  2964 raft_consensus.cc:993] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:28.277469  2686 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.235496  3111 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:28.449178  3105 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:28.444937  2967 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.446827  3043 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:28.465879  3033 consensus_meta.cc:287] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.021s	user 0.004s	sys 0.007s
I20260709 13:13:28.436966  3104 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.444144  3115 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:28.418387  2900 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.472393  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1395) took 1278 ms. Trace:
I20260709 13:13:28.474073  2959 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.439397  3095 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:28.440003  3107 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.452713  1595 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:28.476825   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 115649us
I20260709 13:13:28.465474  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 394) took 1124 ms. Trace:
I20260709 13:13:28.471673  3108 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:28.503544   925 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:28.509609  1003 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:28.502183  1001 rpcz_store.cc:276] 0709 13:13:27.194482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.194760 (+   278us) service_pool.cc:224] Handling call
0709 13:13:28.472233 (+1277473us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54000483d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:28.472338 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845648128}
I20260709 13:13:28.497246  2974 raft_consensus.cc:1081] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:13:28.499398  3104 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.529574  3105 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.512472  3115 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:28.532879  3122 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.084s	user 0.000s	sys 0.002s
I20260709 13:13:28.533432  3115 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.494833  2986 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:28.538115   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.512244  3095 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:28.495044  2818 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:28.537084   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 175283us
I20260709 13:13:28.548094  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.548482   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.549728   819 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:28.550644   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.551529   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.551999  1001 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:28.552641  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.550623   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:28.553478  1000 raft_consensus.cc:2393] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:28.554386   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.555181   998 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:28.523423  1597 rpcz_store.cc:276] 0709 13:13:27.340834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.340944 (+   110us) service_pool.cc:224] Handling call
0709 13:13:28.465303 (+1124359us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400270954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:28.465419 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232661376}
I20260709 13:13:28.556098  3107 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:28.556591  3107 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.557022  3107 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:28.555912  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:28.557344  2967 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.558691  2686 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.554469  3111 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:28.560025   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:28.560336  2900 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.561057   997 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:28.562178  3108 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.557368  2986 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.564684  1000 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:28.570771  2986 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:28.571341  2986 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.571873  3095 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.573132  3095 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:28.574069  2986 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.578765  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:28.584404  2979 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.584239   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:28.585366  3028 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:28.585860  3028 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:28.560191  3111 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.538826  2974 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:13:28.596547  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:28.931730  1598 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:28.607849   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53527.6us
I20260709 13:13:28.934371   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:28.934365   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:28.650827   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.646106  2979 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:28.578683  2998 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.614197  2924 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.586553  2698 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.586598  2818 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:28.973081   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:28.973692   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.645756  2680 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.853752  2901 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.618472  2914 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.864427  3130 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.555915  3124 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:28.865175  2686 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.645421  3127 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.931308  3111 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:28.942692  2986 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.942905  2900 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.207090  2900 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.210078  2900 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:28.939339  2968 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.616274  3018 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.211025  2968 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.212917  3018 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:29.051882   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 78878.1us
I20260709 13:13:29.213651  2968 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.214370  2968 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:28.615151  3088 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.425s
I20260709 13:13:28.863029  3101 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.219205  3101 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.147322  2818 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.220037   927 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.220197  2818 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.975777  2979 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.223899  2979 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.224592  2960 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:29.225073  2960 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:29.226119  2960 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:28.888063  3104 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:28.972496  2965 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.228049  2965 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.148016  3124 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.229044  3124 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:29.229562  3104 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.230373  2965 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:28.977537  2964 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.231736  2964 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.232127  2964 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.147249  2914 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.214939  2968 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:29.311710   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 343285us
I20260709 13:13:29.313356   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:29.031518  2924 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.180068  3127 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.112238  2974 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.149680  3130 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.188098  2686 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.195637  3115 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.975193  3119 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.177s
I20260709 13:13:29.204264  2995 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:28.586645  3028 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:29.253021  3108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.016s	sys 0.017s
I20260709 13:13:29.237622  3093 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.324205  2900 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.112524  2680 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.117038  2901 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.348493  2997 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:29.325569  2914 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:29.353067  2960 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:29.365379  3051 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:29.368213  3108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.017s	sys 0.018s
I20260709 13:13:29.370451  2995 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.373556  2900 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.359380  3115 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.369946  2818 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.358439  2974 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.416399  2974 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.418666  3127 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.421433  2900 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.409828  3130 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.411732  2998 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.436599  2680 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.413729  2995 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.403797  3115 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.426432  2901 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.374778  3093 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.444361  3093 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.459272  2968 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.486678  2924 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.468991   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1404) took 1236 ms. Trace:
I20260709 13:13:29.488559   996 rpcz_store.cc:276] 0709 13:13:28.232068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.232215 (+   147us) service_pool.cc:224] Handling call
0709 13:13:29.468781 (+1236566us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540035aa54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:29.468919 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278609792}
I20260709 13:13:29.470434  1004 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:29.471556   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:29.491077   997 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.472707   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.473599  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.477285  2995 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.496104  2995 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.499289  2995 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:29.469548  1003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1508) took 2096 ms (client timeout 1803 ms). Trace:
W20260709 13:13:29.500375  1003 rpcz_store.cc:269] 0709 13:13:27.373387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.373498 (+   111us) service_pool.cc:224] Handling call
0709 13:13:29.469328 (+2095830us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540034eed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:29.469468 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2587981056}
W20260709 13:13:29.513643  3136 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:13:29.517963  3034 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:29.518584  3034 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:29.519707  3034 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.521127  3034 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.536151  2901 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.536829  2901 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.540288  2901 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.536782  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:29.546198  2901 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:29.546756  2901 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.547220  2901 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:29.551265  3117 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.552192  3117 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:29.468454  3081 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.866s	user 0.032s	sys 0.034s
I20260709 13:13:29.495102  3010 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.622457   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 187) took 1864 ms. Trace:
I20260709 13:13:29.623587   847 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.623514   854 rpcz_store.cc:276] 0709 13:13:27.757937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.758097 (+   160us) service_pool.cc:224] Handling call
0709 13:13:27.758436 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.758657 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.758686 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:27.759152 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.946336 (+187184us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:27.961022 (+ 14686us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.115651 (+154629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.116081 (+   430us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.531671 (+1415590us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b1800449ec0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:29.621437 (+ 89766us) raft_consensus.cc:1685] finished
0709 13:13:29.621635 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.622052 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11312,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":17964,"raft.queue_time_us":23047,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":38784768,"thread_start_us":340561,"threads_started":2,"wal-append.queue_time_us":238403}
I20260709 13:13:29.626713   852 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.629140   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1427) took 2302 ms. Trace:
I20260709 13:13:29.634231   923 rpcz_store.cc:276] 0709 13:13:27.326860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.326963 (+   103us) service_pool.cc:224] Handling call
0709 13:13:27.327218 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.327524 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.327567 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:27.328185 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.456465 (+128280us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:27.456504 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.456573 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.467665 (+2011092us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:29.628353 (+160688us) raft_consensus.cc:1685] finished
0709 13:13:29.628394 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.628847 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1205,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"spinlock_wait_cycles":45441792,"thread_start_us":127877,"threads_started":1,"wal-append.queue_time_us":234360}
I20260709 13:13:29.630281   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.631942   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.639519   926 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:29.641161   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:29.642180   847 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:29.645627  2958 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.646211  2686 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.646662  2958 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.648190  2900 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.648990  2900 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.650594  2958 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:29.698414  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.012s	sys 0.004s
W20260709 13:13:29.703145   970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.005s	sys 0.015s
W20260709 13:13:29.699311   819 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.004s	sys 0.013s
I20260709 13:13:29.672035  2900 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:29.711987   968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.037s	sys 0.089s
I20260709 13:13:29.712435  2997 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:29.647369  2698 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.764052  2698 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.764477  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:29.765065   968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.037s	sys 0.089s
I20260709 13:13:29.765820  1599 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:29.718129  2856 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.720091   852 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:29.767560  2856 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.768954   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:29.768618  2997 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.725508   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:29.769874   847 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:29.770152  2698 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.732846   849 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:29.770727  2856 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.773326   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1589) took 1588 ms. Trace:
I20260709 13:13:29.773983   853 rpcz_store.cc:276] 0709 13:13:28.185101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.185232 (+   131us) service_pool.cc:224] Handling call
0709 13:13:28.185507 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.621162 (+1435655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.621262 (+   100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.621362 (+   100us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.621435 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.760143 (+138708us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000c99d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:29.760189 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.773116 (+ 12927us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3299790592}
I20260709 13:13:29.741643  3010 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.776700  3010 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:29.778878   927 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 13:13:29.742928   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.743651  2686 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.744375   925 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:29.750139   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1435) took 2387 ms. Trace:
I20260709 13:13:29.753455  2959 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.756517   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:29.762094   819 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.005s	sys 0.013s
I20260709 13:13:29.764400  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:29.763084  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.013s	sys 0.005s
I20260709 13:13:29.722913  2964 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:29.761725   970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.005s	sys 0.015s
I20260709 13:13:29.750233   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.781114   927 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:29.780573   929 rpcz_store.cc:276] 0709 13:13:27.362535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.362636 (+   101us) service_pool.cc:224] Handling call
0709 13:13:27.362915 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:28.459516 (+1096601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.749634 (+1290118us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400319a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:29.749949 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520271104}
I20260709 13:13:29.790885  3142 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:29.791169   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.792239   848 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.792560   925 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 1.
I20260709 13:13:29.794721   846 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.795715  2686 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:29.794849  3107 consensus_meta.cc:287] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 1.237s	user 0.011s	sys 0.006s
W20260709 13:13:29.785528  2944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.071s	sys 0.181s
I20260709 13:13:29.782421  2959 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:29.800446  3142 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.800277  3107 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:29.806367   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.806118  2856 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:29.823917  2944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.072s	sys 0.181s
I20260709 13:13:29.854007  2959 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:29.856931  3145 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.075s	user 0.010s	sys 0.002s
I20260709 13:13:29.858330   928 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:29.808403  3028 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.858991   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 213) took 1649 ms. Trace:
I20260709 13:13:29.808523  3094 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:29.860757   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:29.863927  3051 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:29.864305   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.864156  3053 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.864584  3144 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.910830  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.914623   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:29.917795   968 leader_election.cc:336] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Timed out: RequestConsensusVote RPC to 127.31.120.4:33689 timed out after 1.843s (SENT)
I20260709 13:13:29.917209  3144 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.893777   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.917213  2856 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.919466   998 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:29.920042  3144 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.920694  3144 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.921485  3088 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.921309  3144 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.921739  3042 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.922071  3088 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:29.869642  3095 consensus_meta.cc:287] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.296s	user 0.011s	sys 0.008s
I20260709 13:13:29.925750  3033 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.926396  3033 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:29.875749   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.927778  2698 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:29.879294   929 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:29.879819   922 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:29.928745  3033 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.930330  2856 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:29.883059  2965 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.932559   846 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:29.931416  2965 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.933306  2698 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.935130  3107 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:29.826152  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 66536.4us
I20260709 13:13:29.865794  2686 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:29.922626  3051 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:29.887647  3111 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.672s	user 0.006s	sys 0.010s
I20260709 13:13:29.923125  3098 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.923640  3053 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:29.924320  3097 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.924139  2863 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:29.924911   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.901403  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:29.889745  3010 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:29.888736  2979 consensus_meta.cc:287] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.008s
I20260709 13:13:29.927960   921 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:29.878690   926 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:29.928624  3088 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:29.930440  3033 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.933007   927 rpcz_store.cc:276] 0709 13:13:28.209565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.209676 (+   111us) service_pool.cc:224] Handling call
0709 13:13:29.821893 (+1612217us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001f0754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:29.858880 (+ 36987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2311922560}
I20260709 13:13:29.922322  3042 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.916947  3142 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:29.879961  2986 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.007s
I20260709 13:13:30.017248   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.015709   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.957576   930 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:13:30.009541   818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.020s
I20260709 13:13:29.956686   929 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:29.935926   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75433.3us
I20260709 13:13:30.000267   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 117) took 3525 ms. Trace:
I20260709 13:13:30.016295  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.981938   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:30.138880   818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.009s	sys 0.020s
I20260709 13:13:29.955986   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.956799   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:29.985212   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:30.069427  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:29.935897  2698 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:29.944836  3144 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:30.149717  2979 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:29.935338  2965 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.158277  3010 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:30.192574  2856 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:30.193356  3010 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.194768  3142 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.196138   850 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:13:30.169727  3124 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.940s	user 0.013s	sys 0.003s
W20260709 13:13:30.170362  3034 consensus_meta.cc:287] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.649s	user 0.006s	sys 0.010s
I20260709 13:13:30.172106  3042 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:30.172633   922 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:30.174674   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:30.205475   930 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:30.176769   923 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.181591   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:29.940634  3116 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.991263  3143 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:29.893507   997 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:29.936672  3094 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.064903  2924 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.042366  2944 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:30.148741  2900 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.098104   749 rpcz_store.cc:276] 0709 13:13:26.474572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.481212 (+  6640us) service_pool.cc:224] Handling call
0709 13:13:30.000002 (+3518790us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:27.898403 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:27.981600 (+ 83197us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:27.981649 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:13:27.981675 (+    26us) tablet.cc:662] Decoding operations
0709 13:13:27.982766 (+  1091us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:27.982868 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:27.982917 (+    49us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:13:28.021784 (+ 38867us) tablet.cc:701] Row locks acquired
0709 13:13:28.024200 (+  2416us) write_op.cc:260] PREPARE: finished
0709 13:13:28.052246 (+ 28046us) write_op.cc:270] Start()
0709 13:13:28.060555 (+  8309us) write_op.cc:276] Timestamp: P: 1783602808050704 usec, L: 0
0709 13:13:28.063003 (+  2448us) op_driver.cc:348] REPLICATION: starting
0709 13:13:28.174985 (+111982us) log.cc:844] Serialized 2814 byte log entry
0709 13:13:29.362417 (+1187432us) op_driver.cc:464] REPLICATION: finished
0709 13:13:29.800688 (+438271us) write_op.cc:301] APPLY: starting
0709 13:13:29.800876 (+   188us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:29.801210 (+   334us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:29.801410 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:29.802710 (+  1300us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:29.803160 (+   450us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:13:29.803246 (+    86us) write_op.cc:312] APPLY: finished
0709 13:13:29.803838 (+   592us) log.cc:844] Serialized 59 byte log entry
0709 13:13:29.807078 (+  3240us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:29.807401 (+   323us) write_op.cc:454] Released schema lock
0709 13:13:29.807496 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:30.032352 (+224856us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:30.032405 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":20933,"reported_tablets":14,"spinlock_wait_cycles":514432,"child_traces":[["op",{"apply.queue_time_us":437819,"apply.run_cpu_time_us":14470,"apply.run_wall_time_us":51838,"num_ops":7,"prepare.queue_time_us":217966,"prepare.run_cpu_time_us":30085,"prepare.run_wall_time_us":509164,"raft.queue_time_us":828036,"raft.run_cpu_time_us":143195,"raft.run_wall_time_us":705720,"replication_time_us":1299221,"thread_start_us":1448951,"threads_started":4,"wal-append.queue_time_us":269554}]]}
I20260709 13:13:29.989370  3149 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:30.183413   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 198538us
I20260709 13:13:29.948436  2968 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.190474  3044 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:30.192036  2979 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.192714  2974 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.185647  3028 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.197145   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:29.935336  2901 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.202246   928 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:30.156548  2986 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.185742  2959 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.936015  3112 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:30.208659   926 raft_consensus.cc:2393] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:30.211068   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:30.217631   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.220387   848 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:30.196195  3142 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.287003   851 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:30.220556   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:30.221052  3051 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.221294   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:30.222075   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:30.223007  1001 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:30.223922  3010 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.228005   929 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:29.895381  3115 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.234349  3143 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.286266  3112 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.235291  1004 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:30.026940  2995 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.183s
I20260709 13:13:29.891862  3117 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.209277  2686 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.160673  3111 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.322294   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:30.031893  3095 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.188587  3098 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.203382  3097 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.286192   846 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:13:30.332558   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 158499us
I20260709 13:13:30.201162  3124 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.305812   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:30.336803   998 raft_consensus.cc:2410] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:30.337399   927 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:30.323254  1598 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:30.209003  3116 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:30.326782   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.326620   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:30.326669   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54530.5us
I20260709 13:13:30.231910  3094 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:30.323057  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 122763us
I20260709 13:13:30.299877   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:30.344084   854 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:30.345769   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:30.350930   929 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:30.360308   852 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:30.361299   921 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:30.343204  3094 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:30.363394   922 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:30.284561  3149 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.385345  2901 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:30.386250  2901 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.386925  2901 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.388844  2856 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:30.390511   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:30.391392  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:30.392205   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.368258  3095 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.394013  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:30.388913  3053 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.396790  3115 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.400101  3117 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.401353  3143 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.401609  3111 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.401793   855 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:30.342970  3088 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.295428  2944 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.335969  2900 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.300498  2968 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.298977  2924 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.429441  3112 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.429802  2968 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.398380  3124 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:30.404008  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 43917.7us
I20260709 13:13:30.408733   923 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:30.360185   750 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:30.415952   927 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:30.417006  2979 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.436710  2979 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.437547  2901 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.439077  2979 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.328137  2974 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.422546  3088 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:30.430426  2964 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:30.422588  3151 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
I20260709 13:13:30.431783   846 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:30.434758  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.410341   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.442973  1003 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:30.447904   930 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:30.424748  2900 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.451411   849 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:30.469980  1004 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:30.543558  1599 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:30.544803   996 raft_consensus.cc:2410] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:13:30.551757   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 109545us
I20260709 13:13:30.563225  1004 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:30.519510   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:30.592298  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:30.593672  1600 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:30.595252   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.595386   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:30.597951  3155 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:30.550768  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:30.598480  3155 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:30.599049  1594 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:30.546089  3098 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:30.772764   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:30.754210   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 295032us
I20260709 13:13:30.988351   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.989562   995 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:30.821241   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:30.607775   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:30.821919   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:30.771229  1000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:13:30.600984   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:30.768836   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 179316us
I20260709 13:13:30.794559  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:30.619486  3095 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:30.788285  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:30.769573  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.607306  3088 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.599568  3097 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:30.589798  2924 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.842768   846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:30.770839  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.770828  3142 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.222s
I20260709 13:13:30.821429  2968 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.880841  3111 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:30.789585  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:30.770437   998 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:30.764282  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:30.609088  3034 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:30.839551  3112 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:30.835479   895 connection.cc:256] RPC call timeout handler was delayed by -1.630860322999979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:13:31.039945   895 connection.cc:256] RPC call timeout handler was delayed by -1.3784379389999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:13:31.040390   895 connection.cc:256] RPC call timeout handler was delayed by -1.3506764710000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 13:13:30.599545   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.040697   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:30.980373  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.032s	sys 0.053s
I20260709 13:13:31.041376   848 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:30.598945  3155 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:30.590304  2974 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.541819   750 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:30.853260   997 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:30.979493  3028 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:30.838006  3094 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:30.618577  1566 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:30.981585   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:30.992446   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.051712   852 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:13:31.051843   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:30.979871  1564 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.039s	sys 0.083s
I20260709 13:13:30.990981  1000 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:30.920030   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:31.053282  1564 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.039s	sys 0.083s
I20260709 13:13:30.765666  2944 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:30.991703  1593 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:30.993423   853 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:30.964382  3149 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.037665   846 raft_consensus.cc:606] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:31.003940   999 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:31.041555  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.032s	sys 0.053s
I20260709 13:13:31.058635  3142 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:31.058884  1002 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:31.059069  3142 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:31.060041  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.060163  3142 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.061502  3142 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.025883   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.064533   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.065187   995 raft_consensus.cc:2368] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:31.065903   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.066433   995 raft_consensus.cc:2376] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4. Re-sending same reply.
I20260709 13:13:31.067236   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.052342  2968 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.048455   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.067906   750 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:31.069705  2968 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.069411   750 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 2 to 5, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:31.070266  2968 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:31.070655  2968 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.070802   999 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:31.071033  2968 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:30.682737   921 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:31.071777   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.071694  3112 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:31.072096  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 453691us
I20260709 13:13:31.072887  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:31.073943  3112 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.074718  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:31.075793  3143 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:31.076527  3143 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:31.078349  3143 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:31.078850  3143 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:31.038766  3034 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:31.080775  3034 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.082188  3034 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:31.053668  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 867550us
I20260709 13:13:31.053438  1000 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:31.072887   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.083887   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:31.051425   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.057835   846 raft_consensus.cc:614] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:31.060774  1002 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:31.002904  3033 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.090619   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.094305  1001 raft_consensus.cc:2368] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
W20260709 13:13:31.095088   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76035.1us
I20260709 13:13:31.096294   852 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:31.097467  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.098553  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.087353  3028 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.101732   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.104653  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.049947  3111 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.106292   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.106550   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.108453   996 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:31.115450   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.115906  3155 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.116602   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.117772   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.118394  3155 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.105671  3112 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.119637  3112 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.122020  3112 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.129113  3111 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.135241  1000 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.138173  1000 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:31.139537  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.143328   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.143803   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1559) took 1160 ms. Trace:
I20260709 13:13:31.144014   998 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:31.144196   921 rpcz_store.cc:276] 0709 13:13:29.983638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.220174 (+236536us) service_pool.cc:224] Handling call
0709 13:13:31.143569 (+923395us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b5400444dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:31.143693 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168127360}
I20260709 13:13:31.145143   998 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.146443   998 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:31.146752  3156 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:31.147166  3156 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.147467  3156 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:31.154572  1598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 426) took 1396 ms. Trace:
I20260709 13:13:31.089232  3033 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.107036   750 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:31.155059  1598 rpcz_store.cc:276] 0709 13:13:29.758498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.763655 (+  5157us) service_pool.cc:224] Handling call
0709 13:13:31.154303 (+1390648us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540045d8d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:31.154478 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186142464}
I20260709 13:13:31.153250  3096 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:31.158877   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.158210   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:31.159430   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51982.2us
I20260709 13:13:31.159572  2686 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.147524   998 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:31.161315   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1463) took 1204 ms. Trace:
I20260709 13:13:31.166591   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1597) took 1538 ms. Trace:
I20260709 13:13:31.169440   854 rpcz_store.cc:276] 0709 13:13:29.640687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.644479 (+  3792us) service_pool.cc:224] Handling call
0709 13:13:29.645056 (+   577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:30.351352 (+706296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.166279 (+814927us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b540043d354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:31.166528 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1601349504}
I20260709 13:13:31.149227   997 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:31.158192  1003 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:31.168418   930 rpcz_store.cc:276] 0709 13:13:30.060207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.328538 (+268331us) service_pool.cc:224] Handling call
0709 13:13:30.354595 (+ 26057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.470721 (+116126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.161092 (+690371us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b54003cafd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:31.161256 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176548736}
I20260709 13:13:31.181478   852 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.182941   999 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:31.106565  3097 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:31.165473  3094 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:31.237354  1004 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:31.173810  3112 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.150681  3116 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.176478  2686 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.147830  3143 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.243384   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.238547   750 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:31.166235  3096 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:31.176870  3155 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.252053   855 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
W20260709 13:13:31.254544   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.147703  3028 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.228021  3095 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 13:13:31.267305   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 209050us
I20260709 13:13:31.269718   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.254334  3097 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:31.227797  3088 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:13:31.252112  3112 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:31.282027  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 207366us
I20260709 13:13:31.277608   855 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.258719  3155 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.314217   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.322983   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.323711   926 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:31.324198   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.314833   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.324890   850 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:31.285791  2979 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.326016  2979 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.328451   921 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:31.329514   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 413945us
I20260709 13:13:31.329941   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:31.329566  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 223856us
I20260709 13:13:31.329258   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.287878  2968 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.379558   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.406926   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.342960   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.423697   928 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:31.377266   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 61975.5us
I20260709 13:13:31.406527   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.331377   855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:31.425791   853 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:31.361438   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.331841   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.341864   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.389021   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.341739   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.388360   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.434788   927 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:31.428027  1002 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:31.427043   926 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:31.411549  3033 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:31.403631  3097 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.437974   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.426440   855 raft_consensus.cc:606] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:13:31.411224   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.432155   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.434857   930 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:31.330922  3142 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:31.319542  3158 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.406049  3166 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.341428   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.314945  3116 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.474421   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.475438   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.476516  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.476948   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.475358   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.409955  3155 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.358254  3094 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.449723  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.444228   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.478299   923 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:31.280652  3144 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.153s
I20260709 13:13:31.529860  3144 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:31.530380  3144 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:31.446398  3034 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.449966  3028 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.516561  3111 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.515717   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.528180   930 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.516671   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.495359   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.529150  2968 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.430119   925 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.531646  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.344807  3124 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.344969  3098 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.187s
I20260709 13:13:31.531952   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.532472   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.445266   821 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:31.513603   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.478753   855 raft_consensus.cc:614] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:31.503293   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:31.552832   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 116553us
I20260709 13:13:31.575196   921 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:31.511809  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 179696us
W20260709 13:13:31.512446   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.571746   851 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.536341   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.401366  3096 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.433485  3161 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:13:31.531491  3144 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.589767  3158 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.595914   923 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:31.570616   853 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.536095  1002 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:31.451622  3119 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.548756   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.600234   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.549764   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:31.331503   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 46694.1us
I20260709 13:13:31.576941   927 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:31.577415   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.577714   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.571291   926 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:31.456077  3095 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.548251   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1451) took 1606 ms. Trace:
I20260709 13:13:31.605393   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.579348   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.581043   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.498119  2979 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.582320   928 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:31.594239   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:31.589322   819 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.4:33689 timed out after 1.601s (SENT) [suppressed 14 similar messages]
I20260709 13:13:31.451828  2959 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.558050   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.599972   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.603960   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:31.534271  3142 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:31.546934   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.606307   923 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:31.439476  3112 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.578274  3124 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.585690  3028 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.604579  3034 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.638417  3174 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:31.630236   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.687251   926 raft_consensus.cc:2393] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:31.582866  3111 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.600991   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:31.641183   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:31.639698   850 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:31.642899   847 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.657868   851 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:31.455196  3156 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.607419   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.526006  2686 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:31.665841   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 107920us
I20260709 13:13:31.695523   854 raft_consensus.cc:2368] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
W20260709 13:13:31.695958   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.695838  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.664932   855 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.667982   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.695098   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.544154  3094 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.669628   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.669595   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:31.697918   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 90587.5us
I20260709 13:13:31.698194   928 raft_consensus.cc:2376] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6. Re-sending same reply.
I20260709 13:13:31.698493   929 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:31.682927  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.685331   848 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:31.685751  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.699393  1002 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:31.699595  1000 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:31.686365   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.700111  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.700421   995 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.644276  2968 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:31.586938  3169 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.604809   924 rpcz_store.cc:276] 0709 13:13:29.874727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.894705 (+ 19978us) service_pool.cc:224] Handling call
0709 13:13:29.895179 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.895508 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:30.201989 (+306481us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.150014 (+948025us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b18003e8000. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df23a 00007f0d02b49a2a
0709 13:13:31.480164 (+330150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:31.480236 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:31.480303 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.480390 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.480531 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108075,"prepare.queue_time_us":380677,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":158139,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":17065,"spinlock_wait_cycles":8879360,"thread_start_us":482482,"threads_started":2,"wal-append.queue_time_us":1165331}
I20260709 13:13:31.594274  3116 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:31.669168   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 79874.3us
I20260709 13:13:31.709836 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:31.711346  2924 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.712405 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:31.712039  2924 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.696568  1001 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:31.685142  3112 raft_consensus.cc:2764] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:13:31.648222  3158 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:31.704102   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 65795.2us
I20260709 13:13:31.714463  2924 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.698035   926 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:31.691192   850 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:31.722245   923 raft_consensus.cc:2376] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5. Re-sending same reply.
W20260709 13:13:31.715731   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 121478us
I20260709 13:13:31.670077  3171 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:13:31.723969   968 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:31.726390   926 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:31.664883  3095 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.639467  3119 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.738257  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.588313  3166 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.696863  2979 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:31.738948  1001 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:31.666927  2959 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.704698  2968 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.740060  2968 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:31.740533   922 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:31.741211  2959 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:31.741771  2959 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:31.742102  2959 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.743289  1004 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:31.734319   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94506.8us
I20260709 13:13:31.702899  3144 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:31.745201  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 80496us
I20260709 13:13:31.691574  3028 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:31.747807  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.749264   928 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:31.749495   921 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:31.732853  3155 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:31.746240   929 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:31.742482  2959 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:31.762584   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:31.762761  2979 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:31.767547   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:31.761145   850 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.771450   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:31.775449   847 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:31.715092  3169 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.787097   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:31.848932   921 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.848779  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.796589  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.851178  1004 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:31.783977   928 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:31.789183   850 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:31.816896   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 118283us
I20260709 13:13:31.853178   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.790179  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.854481  1001 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:31.794592   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.854933  3158 raft_consensus.cc:2764] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:31.855300   926 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:31.855455  3192 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:31.859047   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 413946us
W20260709 13:13:31.859786   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 97268us
I20260709 13:13:31.856141   929 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:31.865650   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.867440   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.868100   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 152585us
I20260709 13:13:31.870426  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:31.797138  3177 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:13:31.776180  3155 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:31.866828   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 133013us
I20260709 13:13:31.873991  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:31.876005  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:31.880259  1001 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:31.878005   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.877223   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:31.876780  1593 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:31.858960  3180 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:31.856439  3098 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:31.879354  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:31.889750  1599 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:31.859565  3188 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:31.892433   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.760861  3142 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.863257  3149 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:31.898298   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.898175  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:31.899873   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:31.900364   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.904040   921 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:31.904942   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 137520us
I20260709 13:13:31.894276  3180 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.911640  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:31.895232  3098 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.915632   930 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:31.912277   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48339.7us
I20260709 13:13:31.859035  3185 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:13:31.911382  3188 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:31.925478  3028 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.926090  3028 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:31.926431  3028 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:31.926777  3028 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.904232  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:31.867901  3192 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:31.945178   850 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:31.916265  3094 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:31.958178  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:31.912734  3124 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.915063  3098 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.959520   927 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:31.961093  1001 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:31.965111  1600 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:31.966158  3179 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.966446  1601 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:31.966710  3179 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:31.968412   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53165.2us
I20260709 13:13:31.969273   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:31.970170  3178 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:13:31.917148  3142 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:31.915519  3180 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:31.973928   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.973613  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:31.972442   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95963.8us
W20260709 13:13:31.978124   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 77889.2us
I20260709 13:13:31.978859   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.979660  3124 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:31.979748   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.980119  3124 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:31.980520  3124 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.980845   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:31.981920   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:31.984201   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 91852.5us
I20260709 13:13:31.986439   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.987063   928 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:31.987910   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.987998   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.989108   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.989228   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.989730   930 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:31.990538   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.991734   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.993290   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.994340   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.994876   930 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:31.995867   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:31.997764   930 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:31.907202  3149 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.027367  3149 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:32.004536  3095 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:31.972273  2686 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.001181  3116 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:32.042711   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 74571.7us
I20260709 13:13:32.022931   925 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:31.961490  3185 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:32.058135   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80156.9us
I20260709 13:13:32.024212  3166 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.003425   929 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:32.062924   929 pending_rounds.cc:85] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 2
I20260709 13:13:32.032917   853 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:31.962749  3111 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:31.963932  3193 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:32.069620  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:31.971757  3187 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:32.012461  3179 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.006302  3034 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:32.071359   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 153378us
I20260709 13:13:32.063318   929 pending_rounds.cc:107] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:32.074384   929 raft_consensus.cc:2889] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:32.072402   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 87474.5us
I20260709 13:13:31.924129  2979 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.032008  3190 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:32.061906  3184 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:32.079074   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.066792  3116 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:32.059090  3194 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.070165  3193 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.058634  3185 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:32.142903  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.145725  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.071555  3174 raft_consensus.cc:2764] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:32.073660  3187 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.072566  3111 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:32.159065   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 189901us
I20260709 13:13:32.072274  3119 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:32.160002   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 178122us
W20260709 13:13:32.160780   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 101493us
I20260709 13:13:32.015811  3155 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:32.172744   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 93735us
W20260709 13:13:32.222230   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59369.3us
I20260709 13:13:32.082372  3190 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:32.253476   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.081481  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.081836  2686 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.141s
I20260709 13:13:32.160180  3194 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:32.241273  2968 consensus_meta.cc:287] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.013s
I20260709 13:13:32.254729  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.256698  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.078943  3196 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
I20260709 13:13:32.162703  3192 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.156912  3182 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:32.257055   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:32.296890  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.298147   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.253091  3185 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.257902  3187 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:32.312987  2959 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.529s	user 0.000s	sys 0.017s
I20260709 13:13:32.312659  3190 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.293183  1592 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:32.313446  2959 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:32.311455  3144 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.004s
I20260709 13:13:32.295748  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.294201  2968 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.255451  3210 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:32.296784   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.267686  3119 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:13:32.322695  3198 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.325431  3198 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.326546  3124 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.175892  3193 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.222914  3184 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.302783  3183 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:32.304999  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.306775  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:32.336313  1002 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:32.260063  3111 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.265067  3028 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.311689  3208 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:13:32.222388  3116 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.367694  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.367964  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.369047  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.369148  1600 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:32.369673  1594 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.370527  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.372002  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.311937  3196 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.312109  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.317545   928 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:32.294296   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.303692  3182 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.321746  2968 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.324260  1001 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:32.269052  3194 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:32.329996   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 72998.8us
I20260709 13:13:32.336485   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:32.341291  3210 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:32.341831  3207 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:13:32.391896   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.335351  3183 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.344570  3119 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:32.327555  3198 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.349536  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.353021  3028 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.354334  3193 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.318744  2959 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.359702   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:32.359896  3098 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:32.082082   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:32.363513  1000 raft_consensus.cc:2410] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:32.364691  3181 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:32.365260  3111 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.366101  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.409090  3098 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.409196  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:32.410156  3199 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:13:32.366969  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.411194  1003 raft_consensus.cc:2410] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:32.412501  3206 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:32.411168  1596 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:32.416903  3193 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:32.417644   738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 351128us
I20260709 13:13:32.419256   846 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:32.420073   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.419907  3210 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.420642   849 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:32.376387  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:32.377475   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83253us
I20260709 13:13:32.369261  3116 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.424103   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.424721  3124 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.382786  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.426836   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.379902  3204 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.383085  3187 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.392052   995 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.343133  3184 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.394198  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.380615  3182 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.375187  3196 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:32.398519   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:32.399397   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 102698us
I20260709 13:13:32.384078  3142 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:32.392525  2686 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:13:32.397500  3180 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.395416  3119 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.395398  3183 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.496476   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.421871   855 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:32.422547  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.403141  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:32.391201   821 connection.cc:256] RPC call timeout handler was delayed by -1.063074239000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
I20260709 13:13:32.431962   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:32.497963  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75508.2us
I20260709 13:13:32.432121  3094 raft_consensus.cc:2764] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:13:32.498694  1004 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:13:32.499116   821 connection.cc:256] RPC call timeout handler was delayed by -1.0522742390000985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:13:32.504406   821 connection.cc:256] RPC call timeout handler was delayed by -1.0396742390000782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:13:32.503849   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 71988.1us
W20260709 13:13:32.504776   821 connection.cc:256] RPC call timeout handler was delayed by -1.0207742390000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:13:32.483917  3098 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.507350  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:32.507532   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 109042us
I20260709 13:13:32.511802   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.508832   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.483306  3142 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.441071  3206 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.514806   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.536863   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.539340   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.546494  3168 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
W20260709 13:13:32.569940  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 142496us
I20260709 13:13:32.551928   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.552255  3124 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:32.552475  3204 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.552919   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.572223   855 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:32.527446  3098 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.555989  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.575011   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.556829  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.556627  3119 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.557713  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.558470  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.498651  3209 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.584071  3206 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.584699  3209 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:32.585175  3209 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.586246   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.586994   847 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:32.559790  3094 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:32.441246  3174 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:13:32.596240  3119 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:32.597452  3156 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.598646   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.600172   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.601397   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.602504   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.559998  3097 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:32.561015   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 60592.9us
I20260709 13:13:32.562856   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.527932  3142 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.603581   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.606851   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.564592  3211 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:32.565321   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.607586   847 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:32.567293  3215 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:32.610478  3215 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.611541  3215 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.612309   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1631) took 1257 ms. Trace:
I20260709 13:13:32.567521   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:32.614290   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46861.9us
I20260709 13:13:32.615329   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.616469  3193 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.616829  3193 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:32.568022   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71691.2us
I20260709 13:13:32.568835  3180 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.569303  3171 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:32.547937  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.575698   849 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:32.576156   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.576403  3098 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.577203   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.575361  1598 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:32.578037   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.575304  3124 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.576576  3214 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.583738  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.586901  3096 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.559544  3195 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:13:32.608086   821 leader_election.cc:336] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.665s (SENT)
I20260709 13:13:32.605335   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.604166  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.580775  3181 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:32.577652  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.617167   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.620056   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:32.623476   848 raft_consensus.cc:2393] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:32.624693   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:32.625504  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46440.4us
I20260709 13:13:32.642560  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.627511   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.644392   995 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:32.645149   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.645756   846 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.612759   851 rpcz_store.cc:276] 0709 13:13:31.355368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.696985 (+341617us) service_pool.cc:224] Handling call
0709 13:13:31.706564 (+  9579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.743520 (+ 36956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.743554 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:31.759986 (+ 16432us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.040768 (+280782us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:32.040833 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.611894 (+571061us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b54002b8254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:32.611950 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.612000 (+    50us) raft_consensus.cc:1685] finished
0709 13:13:32.612017 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.612249 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84349696,"thread_start_us":193055,"threads_started":1,"wal-append.queue_time_us":285120}
I20260709 13:13:32.605031   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.627660   853 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:32.634665   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.646026   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.638808   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.653098   855 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.654578   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.634472  3195 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:32.656874  3195 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.651365  2686 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:13:32.636301  1593 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:32.617198  3193 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.632817  1596 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:32.665877   855 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:32.650753   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.667932   853 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:32.651710  3182 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.635742   852 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:32.669773   850 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:13:32.672576  3182 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.674046   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:32.661397  3098 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.676299   999 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:32.678102   851 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:32.673907   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.680382   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.689874   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.692011   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.620327  2959 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.625754  3124 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.698491   896 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.605523  3097 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:32.618608  3171 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.689031   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:32.700714   847 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:32.696254   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.655547  3144 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:32.625798  3169 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.674868  3180 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:32.630126  3187 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.675356  3211 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.636504  3181 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.711572   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:32.634470  1594 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
W20260709 13:13:32.715162  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 72644.3us
I20260709 13:13:32.620514  3192 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:13:32.621137  3095 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:13:32.617357  3198 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
W20260709 13:13:32.717816   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 112862us
I20260709 13:13:32.688143  3206 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.722116  1599 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:13:32.722214  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.652266  3185 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.702692  3156 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.686877  3112 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:32.704668  3096 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:32.705241  3182 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.635457  3194 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.702788  3033 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:32.729439   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:32.705540  3171 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:32.706585  3116 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:32.760700   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.713215  3208 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:32.719576  3180 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.627833  2979 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 13:13:32.732213   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 115098us
I20260709 13:13:32.718672  2959 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.718487  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.720216  3210 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:32.628474  3183 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.720387  3143 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:32.719134  3211 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:32.625171  3196 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.632556  3214 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.734103  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.737540  3098 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:32.740702  3097 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.747932  3167 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:32.808944  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.748934  3194 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.748994  3212 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:32.750923  3171 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.753494  3190 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.757894  3033 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.749608  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.741868   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:32.822284   854 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:32.822861   854 pending_rounds.cc:85] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 2
I20260709 13:13:32.823177   854 pending_rounds.cc:107] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:32.760160  2686 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:32.824241   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.825101   924 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:32.825183   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83385.9us
I20260709 13:13:32.733129  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.733318  3187 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.773784   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.732225  3174 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:13:32.773047  3180 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.784291  1592 raft_consensus.cc:2376] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6. Re-sending same reply.
I20260709 13:13:32.824913  3212 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:32.831970  3171 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.832513  3098 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:32.832976  3098 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.832789  3212 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.769826  3208 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:32.821966  3033 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.835809  3208 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:32.835961   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 62284.5us
I20260709 13:13:32.836720   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.837601   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:32.790987   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.839802  3194 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:32.839890   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:32.840413  3194 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.783339  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:32.841336  3194 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.798554   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:32.799971  3196 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.842871  3214 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.843500  3214 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.844717  3111 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:32.801800  3185 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.803372   850 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:32.776101  3116 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:32.812426  1601 raft_consensus.cc:2368] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:32.806926  2959 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.782881  3169 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:32.781975  3210 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:32.817088  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:32.848829  3183 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:32.849586  3210 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.787446  3211 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.851876  3116 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.852516  3096 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:32.852988  3096 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:32.853850  3096 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:32.842885   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44421.9us
I20260709 13:13:32.858235   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:32.858892   928 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:32.860397  3214 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.861447  3214 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.862663   895 leader_election.cc:400] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
I20260709 13:13:32.862987   895 leader_election.cc:403] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:32.865394  3225 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:32.865924  3225 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.866120   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:32.866837   921 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:32.866771  3167 consensus_queue.cc:1059] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:32.823501   854 raft_consensus.cc:2889] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:32.828761 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:32.868077  3167 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:32.868394  3143 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:32.868535  3167 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:32.868940  3143 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.869495  3167 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.870872  3167 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.833415  3098 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.787519  3168 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:32.871996  3181 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.872500  3168 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:32.872867  3168 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:32.873703  3168 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.874181  3181 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:32.874840  3181 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
I20260709 13:13:32.874969  3168 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.876086   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:32.876874   850 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:32.839921  2979 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.866124   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:32.865512   999 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:32.889127   923 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:32.733333  3144 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:13:32.850735  3210 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.828128  2686 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:32.820688  1564 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.866333  3228 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:32.825230  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 91174.7us
I20260709 13:13:32.891311   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:32.897619   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.907084   928 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.898943   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.907886   929 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.899047   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.899485   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:32.795363   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 213007us
W20260709 13:13:32.912835   821 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.803s (SENT)
I20260709 13:13:32.913410   821 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:32.901942   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:32.877432  3224 log.cc:927] Time spent T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.147s	user 0.000s	sys 0.004s
I20260709 13:13:32.824662  3156 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:32.923235   923 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.925348   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.926028   930 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:32.827404  3206 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.928431   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:32.921083   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:32.930330   922 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:32.930967   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:32.935424  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 114804us
W20260709 13:13:32.946974   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 227237us
I20260709 13:13:32.954365   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:32.914836  2686 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.925197  3156 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.955674  3175 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:33.039402   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85138us
I20260709 13:13:33.097218   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:33.098614   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:32.928087  2979 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:32.935134  3119 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:32.822464  3209 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.900736  3193 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:32.942994  3192 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:32.905059  3124 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:32.866788  3225 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:33.123433   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 247328us
I20260709 13:13:33.123697   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.124034   923 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:33.128443   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.156293   999 raft_consensus.cc:2410] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:33.135092   928 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:33.137200   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.036674  3233 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.168610   995 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:33.138743   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:33.096129  3156 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.926795  3228 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.170993   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:33.099267  3215 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.176586   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:33.177228   930 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:33.165505  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.178695  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.186318  1003 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:33.179656   927 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:33.180181   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:33.187929   928 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:33.180388  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.181005   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:33.181222   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 268624us
I20260709 13:13:33.194175   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:33.193778   998 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:33.195998   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.208891  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.212845  1004 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:33.213337   923 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:33.139628  3192 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:33.214484   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.202037   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.217453   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.130340  3142 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:33.202847  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 266218us
I20260709 13:13:33.200600   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.184357  3119 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.220412  1564 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:33.220434  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.177346  3233 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:33.183933  3215 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.204890   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:33.227255   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.216856   928 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:33.193257  3234 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:33.229688   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.199899  3182 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:33.233122   999 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:33.233495   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:33.230909   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 287397us
I20260709 13:13:33.231122   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.231805   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.235380   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.236943   894 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:33.239355   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.242030   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.242862   922 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:33.243880   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.244623   922 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:33.242743   923 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:33.247974  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.217240  3192 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.250113  1002 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:33.174387  3206 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.255420   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:33.256382   922 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:33.224634  3225 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:33.257371  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.225816  3233 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.199859  2959 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:13:33.232244  3215 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.263134  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.267558  1004 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:33.265059   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:33.268620   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.266074   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.265347   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.269714   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:33.268666  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 48374.8us
I20260709 13:13:33.270813   923 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:33.271764   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.234581  3116 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.241549  3179 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.264396   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:33.248360  3169 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.243221  3111 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.229585  3212 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:13:33.292022   929 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:33.278646   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.278475   922 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.290365   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.273473  3235 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:33.279495  3198 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:33.276548  3187 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:33.318427  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.287221  3169 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:33.321887  1004 raft_consensus.cc:2376] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5. Re-sending same reply.
I20260709 13:13:33.322599   924 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:33.327960  3112 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.252408  2979 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.292078  3182 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:33.333875  3182 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.349407   925 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:33.349242  3143 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:33.338967   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.300464  3116 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:33.357220   926 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:33.363327  3143 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:33.363759  3230 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 13:13:33.368320   819 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Timed out: RequestConsensusVote RPC to 127.31.120.4:33689 timed out after 1.601s (SENT)
I20260709 13:13:33.370445   926 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:33.319860  3155 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.307492  3235 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:33.324710  3187 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.339123  3169 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.353922  3175 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.331239  3248 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:13:33.407756  3235 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.409102  3258 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:33.409827  3175 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.409556  3233 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.410463  3175 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:33.381354   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 111720us
I20260709 13:13:33.414548   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.416113   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.417551   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.381994  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.419817  1004 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:33.384644  3253 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:33.387279  3193 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:33.390025  3124 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:33.390266  3033 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.554s	user 0.004s	sys 0.003s
I20260709 13:13:33.392330  3209 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:33.392907  3171 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.000s
W20260709 13:13:33.393962  3208 consensus_meta.cc:287] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.001s
I20260709 13:13:33.394680   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.378516  3187 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.398286  2686 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.400996   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:33.395551  3098 consensus_meta.cc:287] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.002s
W20260709 13:13:33.408025  3196 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.000s
I20260709 13:13:33.409278  3235 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.435074   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:33.409574  3258 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:33.443167  3033 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.441666  3116 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:33.445026  3167 consensus_meta.cc:287] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.574s	user 0.017s	sys 0.000s
W20260709 13:13:33.380896  3190 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.004s
W20260709 13:13:33.409844  3239 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:13:33.497778  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.498829  1003 raft_consensus.cc:2368] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:33.452397   999 raft_consensus.cc:2376] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3. Re-sending same reply.
I20260709 13:13:33.453254  1600 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:33.455389  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:33.442231  3180 consensus_meta.cc:287] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.005s
I20260709 13:13:33.460104   925 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:33.440513   922 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:33.513571  1000 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:33.460119  3098 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.462688  3233 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.463008  2686 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.464972   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:33.517517   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52613.7us
I20260709 13:13:33.518033   997 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:33.505751  1567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.011s
W20260709 13:13:33.468389  3210 consensus_meta.cc:287] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.000s
I20260709 13:13:33.468277  3246 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.519228  3246 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.519637  3246 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.519763  3111 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:33.515646  3142 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.469280   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:33.523752  3213 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.524060   995 raft_consensus.cc:2368] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:13:33.477412  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.524999  1004 raft_consensus.cc:2376] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5. Re-sending same reply.
I20260709 13:13:33.525249  1001 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:33.479770  3112 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:33.480748  3255 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.526119  3112 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:33.526562  3112 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.527587   921 raft_consensus.cc:2410] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:33.490310  3179 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.493357  3258 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.454097  3234 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:33.503041  3215 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.518548   895 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.537902  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:33.538995   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:33.518600  1567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.007s	sys 0.011s
W20260709 13:13:33.548121  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 622123us
W20260709 13:13:33.490151  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 210794us
I20260709 13:13:33.554378  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:33.572080   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 195251us
I20260709 13:13:33.577207   848 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:33.413829  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 117393us
I20260709 13:13:33.582394   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:33.583209   848 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.456744  3208 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.586822   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:33.590888   930 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:33.593947   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:33.595135   853 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:33.446748  3253 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.598897  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:33.599413   846 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:33.579571  2686 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:33.580572  3255 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.612949   849 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:33.611680   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:33.606009   851 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:33.603439   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 292597us
I20260709 13:13:33.517439  3180 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.623977   996 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:33.625204  3180 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.591097  3167 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:33.591115  3111 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.554931  3179 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.632344   854 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:33.636574   847 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:33.598793  3210 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:33.639914  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 54139.4us
I20260709 13:13:33.585690  3213 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:13:33.646960  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:33.650281   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:33.633850  3111 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.653615  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:33.653702  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:33.654421  1596 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:33.654950  1595 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:33.653635  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.657007  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.659128  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.660401  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.664448  3241 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:33.633983  3180 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.653926  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:33.681190  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 81203.6us
I20260709 13:13:33.692262  3195 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:33.693667   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:33.694257   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:33.699050   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 48839.2us
I20260709 13:13:33.701112   894 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.702620   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.704475   894 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.647125  3033 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.639655  3262 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:33.710907   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 61114.6us
I20260709 13:13:33.717782   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:33.665015  3208 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.708956   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:33.725692  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.724184  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:33.724190  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:33.646241  3194 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:13:33.727540  3194 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:33.728013  3194 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:33.633376  3175 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:33.729012   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.732100  3196 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.732813  3196 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.735291  1598 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:33.735445  3196 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.732060  3212 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.736428  3212 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.739123  3212 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.731243  3175 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.733741  3194 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.643174  3156 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.766111   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:33.698235  3255 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:33.754580   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:33.813527   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 109103us
I20260709 13:13:33.755355  1599 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 7: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:13:33.756058   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 47194.3us
I20260709 13:13:33.664999  3241 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.646878  3214 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
W20260709 13:13:33.762705  3211 consensus_meta.cc:287] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.903s	user 0.008s	sys 0.004s
I20260709 13:13:33.766731   851 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.722174   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.706578  3116 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.706076  3144 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.692350  3169 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.698563  3180 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:33.724663  3168 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.001s
I20260709 13:13:33.764679  3212 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.969667   848 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:33.813261  3181 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.818553   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:33.873306   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 151187us
I20260709 13:13:33.691431  3210 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:33.724874   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:33.989398  3195 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:33.989689   738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 270708us
I20260709 13:13:33.757170  3194 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.990824  3210 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.951781  3119 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.649524  3234 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.992617   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:33.993747   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:33.995536  3265 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.995900  3263 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:33.724924  3206 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.236s
I20260709 13:13:33.996354  3263 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:33.996755  3263 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.997416  3215 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.816759  3196 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.000149   924 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:34.001216   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.001559   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.002358   853 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:34.003942   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.005110   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.005859   849 raft_consensus.cc:2410] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:34.006678   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.007937   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:33.982373  3212 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.014695  3263 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.935000  3256 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.017452  3263 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.869027  3255 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:34.017922   848 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:34.019398  3255 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:33.874792  3211 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.024917  3211 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:33.974268  3155 consensus_meta.cc:287] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.000s
I20260709 13:13:34.026135  3155 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.961336  3144 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.029177  3144 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.032001   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1602) took 1125 ms. Trace:
I20260709 13:13:34.032410   923 rpcz_store.cc:276] 0709 13:13:32.906603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.273043 (+366440us) service_pool.cc:224] Handling call
0709 13:13:33.273317 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.593622 (+320305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:33.596612 (+  2990us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:33.601805 (+  5193us) log.cc:844] Serialized 26 byte log entry
0709 13:13:33.712730 (+110925us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:33.712834 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.712930 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.031762 (+318832us) raft_consensus.cc:1685] finished
0709 13:13:34.031780 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.031938 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22958,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":13564,"thread_start_us":363083,"threads_started":2,"wal-append.queue_time_us":395847}
I20260709 13:13:34.033437  3180 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:33.935925  3269 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:13:33.957573  3156 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:33.950115  3098 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:33.981563  1601 raft_consensus.cc:2410] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:34.039391  3212 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:33.982021   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 163529us
I20260709 13:13:34.048162  3194 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.048925  3194 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.001232  3169 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:33.951431  3192 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.000s
I20260709 13:13:33.933869  3112 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:33.754746  3182 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:33.814409  3167 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:33.955744  3179 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.003259   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.059345   853 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:33.824730  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 445) took 1046 ms. Trace:
I20260709 13:13:34.072464   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.073933   849 raft_consensus.cc:2410] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:34.066037  1600 rpcz_store.cc:276] 0709 13:13:32.780922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.782957 (+  2035us) service_pool.cc:224] Handling call
0709 13:13:33.824545 (+1041588us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540063f454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:33.824669 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479580032}
I20260709 13:13:34.076215   922 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:33.823591  1593 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:13:34.094612   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49971.5us
I20260709 13:13:34.095762   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.060801  3167 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.118131  3112 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.118404  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.118852  3179 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:34.119360  1001 raft_consensus.cc:2410] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:13:34.071832  3187 consensus_meta.cc:287] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 13:13:34.121488  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.122506  1004 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:34.123412   851 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.119431  3179 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.134012  1001 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:34.133544  3194 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.135792  3179 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.136652  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.136803   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.058596  3192 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.137588  1004 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:34.137966   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.138794  1004 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:34.138968   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.139127  3192 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.139262  3098 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.954556  3246 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.139892  3098 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.141549  3246 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.142498  3098 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.143195   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:34.144258   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.143206  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.144778   849 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:34.145285   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.143482   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.143488   850 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:34.142144  3187 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.135460  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.154918  1003 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:34.160828  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.161439  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.163595   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.155478  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1552) took 1014 ms. Trace:
I20260709 13:13:34.156322   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.165891   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.166460   851 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:34.167224   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.164613  1002 rpcz_store.cc:276] 0709 13:13:33.140921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.165127 (+ 24206us) service_pool.cc:224] Handling call
0709 13:13:34.154618 (+989491us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54000483d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:34.155396 (+   778us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144645888}
I20260709 13:13:34.170433   849 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:34.172634   849 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:34.180491   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1590) took 1322 ms. Trace:
I20260709 13:13:34.180969   927 rpcz_store.cc:276] 0709 13:13:32.857577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.857700 (+   123us) service_pool.cc:224] Handling call
0709 13:13:32.899807 (+ 42107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.187448 (+287641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.180210 (+992762us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54000019d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:34.180424 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642643840}
I20260709 13:13:34.184247   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.185035   995 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:34.191216   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.194819   923 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:34.195451  1566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.012s	sys 0.023s
W20260709 13:13:34.196036  1566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.023s
I20260709 13:13:34.198276   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.200562   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.202258   849 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:13:34.203892   849 pending_rounds.cc:85] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:34.204236   849 pending_rounds.cc:107] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:13:34.204586   849 raft_consensus.cc:2889] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.201287   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.195514   853 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:34.208850  1564 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.210111   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.211050   851 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:34.211128   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.212016   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.216236   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.208062  3112 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:34.220687  3156 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.221678  3156 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:34.223799  1594 consensus_meta.cc:287] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.003s
I20260709 13:13:34.224339  1594 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:34.225783  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.218585  3112 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:34.228008   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.228760   849 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.230436   969 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:34.248878  3272 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:34.249855  3272 consensus_queue.cc:1243] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:34.253197   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.266070   851 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:34.227795  3112 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.288199   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:34.286298  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.289535   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:34.291136  3098 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:34.291790  3098 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:34.292922  3098 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.295732  3246 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:34.296306  3246 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.296762  3246 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.218602  3262 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 13:13:34.313356  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 99799.2us
I20260709 13:13:34.314218   995 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:34.264335  3272 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:34.314347   927 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:34.313366  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.317157  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.318718  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.285226  3095 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:13:34.320643  3187 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.321235  3280 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:34.268103  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.323050   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.319967  3241 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.332099  3256 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:13:34.325960  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 117180us
I20260709 13:13:34.285226  3169 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.341028   995 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:34.325908  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.285233  3279 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.343357  1004 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:34.349165  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.321637  3280 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:34.322054  3272 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:34.354416   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:34.358876   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 102360us
I20260709 13:13:34.348170  3241 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:34.368852   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.373981  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.369195   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.385094   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1492) took 1257 ms. Trace:
I20260709 13:13:34.350332  3279 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.351744  3169 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.417111   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.419157  3279 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:34.419534  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45649.6us
I20260709 13:13:34.420409   850 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:34.421366  1595 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:34.421960  3174 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:34.422371  3253 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.416877  3211 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.422446  3174 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.423277  3211 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.424563  3171 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:34.424685  3174 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.425143  3171 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:34.425305  3211 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.362147  3198 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:34.426034  3286 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.426133  3171 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.427618  3169 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.392649   849 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.365123  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.375020  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:34.429452   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.359099  3246 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.373795  3277 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:13:34.400887  3204 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:34.402467  3272 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.394136  1002 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:34.405364  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.449705  1601 raft_consensus.cc:2368] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 6.
I20260709 13:13:34.429415  1597 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:13:34.432456   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 63341.3us
I20260709 13:13:34.426034  3198 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:34.436235  3264 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.431684   925 rpcz_store.cc:276] 0709 13:13:33.127916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.350256 (+222340us) service_pool.cc:224] Handling call
0709 13:13:33.350528 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.513577 (+163049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.384741 (+871164us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001699d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:34.385015 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251596160}
I20260709 13:13:34.442764   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.400511  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.446682   821 leader_election.cc:400] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.447579   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.357506  3155 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:13:34.450330   847 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:34.433094   848 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:34.453191   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.454475  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.459818   821 leader_election.cc:403] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:34.462220  1002 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:34.445564  3034 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.445348  3214 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:34.465375  3264 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.465651  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.466050   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.466599   848 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.467417  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.469172   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.471273   849 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:34.472473  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.474215  1601 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:34.475236   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.475337  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.455251  3246 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.374505  3280 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:34.475253  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 135435us
I20260709 13:13:34.481761   847 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:34.457959  3248 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:13:34.464422  3279 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.484637  3264 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.452760  3204 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.485417  3279 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.487123  3204 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.482025  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.470002  3034 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.474366  3234 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.491667   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:34.492690  3034 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:34.494338   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:34.496385   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:34.482715  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.497247   999 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:34.497166  1002 raft_consensus.cc:2376] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5. Re-sending same reply.
I20260709 13:13:34.500555   894 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.477167  3192 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.455713  3212 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:13:34.504338  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.507517   821 leader_election.cc:400] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:13:34.510721   848 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:34.502831   894 leader_election.cc:400] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 6.
I20260709 13:13:34.529565  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.534389   927 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:34.501489  3169 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:34.494837  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.498548  3234 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:34.530416  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:34.490314  3267 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:34.546355  1002 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:34.548084  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.549142  1002 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:34.503557  3253 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:34.552273   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44944.2us
I20260709 13:13:34.552980   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.553015  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.509018  3206 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.553843  1002 raft_consensus.cc:2368] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.527230  3215 raft_consensus.cc:2764] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:13:34.546689  3155 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.570989  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.504036  3204 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:13:34.573807  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.574772  1002 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:34.554075  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.559976  3169 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.579023   927 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:34.571813  3155 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.581351   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:34.582398  3111 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.578846   925 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:34.568367  3250 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:34.590317   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.590950   925 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.587970   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.560225  3234 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:34.591838   927 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.599891   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.600041   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.592832   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.600600   849 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:34.600939   928 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:34.589962  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.599295   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.577554  3171 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.596871   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:34.595983  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 116363us
I20260709 13:13:34.599393   848 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:34.608253   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.603993   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.603420   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:34.595794   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:34.637729   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:34.637684   848 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:34.640793   849 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:34.569025  3292 consensus_queue.cc:1059] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:13:34.645434   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.646250  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.646301   928 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.602504  3111 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:34.641139   848 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.609841   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.649135   926 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.607820  3214 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:34.650286   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54574us
I20260709 13:13:34.589135  3192 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:13:34.610219   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.653963   930 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:34.653326   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.658221   927 raft_consensus.cc:2410] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:34.659197   848 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:34.659848  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57016.3us
I20260709 13:13:34.660054   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.660780  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.645392  3263 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.638041  3250 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:34.598003  3169 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.672933   928 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.680192   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.680768   928 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.681510   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.682029   928 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:34.682724   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.683228   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:34.684274   928 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:34.661324   929 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:34.676757   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.679024   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.686120   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.687501   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.688134   930 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.649897  3111 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:34.691675   925 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:34.692700  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:34.706070   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.708422   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.655745  3300 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.712108   848 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:13:34.714603   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1553) took 1570 ms. Trace:
I20260709 13:13:34.722723   848 pending_rounds.cc:85] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 2
I20260709 13:13:34.723047   848 pending_rounds.cc:107] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:34.723104   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1555) took 1578 ms. Trace:
W20260709 13:13:34.717319  3297 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.137s	user 0.002s	sys 0.000s
W20260709 13:13:34.677824  3294 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:13:34.608296  3112 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.723347   848 raft_consensus.cc:2889] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.726544   926 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:34.721993   996 rpcz_store.cc:276] 0709 13:13:33.143743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.143865 (+   122us) service_pool.cc:224] Handling call
0709 13:13:33.178572 (+ 34707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.624323 (+445751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.714309 (+1089986us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400046ad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:34.714517 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023886720}
I20260709 13:13:34.718238   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1490) took 1590 ms. Trace:
I20260709 13:13:34.750440   924 rpcz_store.cc:276] 0709 13:13:33.127367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.331083 (+203716us) service_pool.cc:224] Handling call
0709 13:13:33.331662 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.000592 (+668930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.717988 (+717396us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400169ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:34.718181 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536803712}
I20260709 13:13:34.750974   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:34.723511   997 rpcz_store.cc:276] 0709 13:13:33.144265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.144411 (+   146us) service_pool.cc:224] Handling call
0709 13:13:33.179088 (+ 34677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.518426 (+339338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.722854 (+1204428us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54003e3354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:34.723027 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779050240}
I20260709 13:13:34.751966   996 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 3: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:34.752298   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.753098   997 raft_consensus.cc:2410] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:34.707408  3034 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:34.757296   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.759882  3111 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.761332  3111 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.764268  1594 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:34.707808  3301 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:34.766776  3155 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.771211  3250 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.694299  3212 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:34.772994   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:34.677565  3263 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.688172  3267 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:13:34.721534  3300 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:34.781838   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:34.783361  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 110611us
I20260709 13:13:34.747509  3228 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:34.787279   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 99201.4us
I20260709 13:13:34.788753   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.789824   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.788223   997 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:34.792399  3155 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.792973  3301 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:34.711287  3234 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.795665  3210 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:34.798620  3247 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:34.799237  3247 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:34.775046  3181 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.788516  3300 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.818279  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.818668   924 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:34.819089  1594 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:34.820333   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.821182   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.821834   997 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:34.779505  3307 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:34.825044   821 leader_election.cc:400] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.825383   821 leader_election.cc:403] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:34.826268   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:34.827003   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.828485   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.798822  3198 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.833428   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.835960   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:34.837131   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.824132  3155 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.788173  3187 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:13:34.840065  3214 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.832947  3234 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.834239  3192 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.843160  3198 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.827971  3307 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:34.839039   997 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:34.832661  3215 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:34.838227  3298 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:34.846545  3111 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:34.832460  3210 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.849217  3263 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:34.832856  3181 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.849028  3187 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:34.846567  3112 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.849851  3247 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.863332  3192 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:13:34.851428  3215 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.869701   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.854007  3111 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:34.864943  3187 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:34.892999  3187 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.893654   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:34.894179  3187 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.894253   999 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:34.871306  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1550) took 1731 ms. Trace:
I20260709 13:13:34.895749  1000 rpcz_store.cc:276] 0709 13:13:33.140290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.140426 (+   136us) service_pool.cc:224] Handling call
0709 13:13:34.871046 (+1730620us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b540022c4d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:34.871232 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738725632}
I20260709 13:13:34.897073  1000 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:34.898280  1000 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:13:34.898548  1000 pending_rounds.cc:85] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 3
I20260709 13:13:34.898736  1000 pending_rounds.cc:107] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:13:34.898936  1000 raft_consensus.cc:2889] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.871446  3192 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:34.886730   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:34.900058  3192 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:34.901118   999 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:34.901463   999 pending_rounds.cc:85] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 1
I20260709 13:13:34.901791   999 pending_rounds.cc:107] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:34.880697  3272 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.880462  3307 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:34.861581   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:13:34.902525   738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 47449.6us
I20260709 13:13:34.902554  3307 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.883675  3169 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.885468  3210 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.885918  3181 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:34.904652  3181 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:34.904994  3181 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:34.905732  3181 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.906772  3181 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.906924   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.907491   928 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:34.886739   852 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:34.907945   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.882268  3302 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:34.890158  3300 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.908682  3302 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:34.909040   930 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:34.909363   927 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:34.875885  3111 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:34.909621  3302 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:34.910288  3300 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:34.910445   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.900967  3192 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.910903  3302 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:34.911464  3192 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:34.901505  3298 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.865393  3198 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:13:34.913285  2959 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:34.913633  2959 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:34.914377  2959 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:34.915431  2959 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.917326  1000 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:34.919793   926 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:34.924840   997 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:34.924986  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:34.926388   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.927336   926 raft_consensus.cc:2368] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:34.927997   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.928221   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.928963   925 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.929355   928 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:34.930052   997 pending_rounds.cc:85] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 2
I20260709 13:13:34.856235   821 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.923187  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:34.932866   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.930501   928 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:34.938791   928 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:34.902109   999 raft_consensus.cc:2889] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.941547   928 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:34.942695  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:34.942615  3307 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:34.943509  1598 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:34.925074   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:34.945127   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:34.930475   997 pending_rounds.cc:107] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:34.947579   997 raft_consensus.cc:2889] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.941457   925 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:34.947479   921 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1494) took 1819 ms (client timeout 1554 ms). Trace:
W20260709 13:13:34.948907   921 rpcz_store.cc:269] 0709 13:13:33.128387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.131210 (+  2823us) service_pool.cc:224] Handling call
0709 13:13:34.947280 (+1816070us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400319a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:34.947412 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731162880}
I20260709 13:13:34.952795   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.953509   927 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.954948   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.956655   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:34.960731   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:34.874975  3215 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:34.950814  3251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.020s	sys 0.058s
I20260709 13:13:34.971959   928 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:34.976121   921 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:34.973928   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:34.978279   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 122109us
W20260709 13:13:34.981159  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.961192  3300 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:35.000576   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76275.1us
I20260709 13:13:35.013264   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:34.973511  3214 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.008797   927 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:34.931043  3298 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:13:34.954350  3262 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:13:34.980154  3234 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:34.997517  3251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.021s	sys 0.059s
I20260709 13:13:35.009927  3300 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.010259  3112 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:35.063086  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.036180  3188 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.066531  3300 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:35.068588  3270 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.407s	user 0.003s	sys 0.009s
I20260709 13:13:35.036505  3210 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:35.075199  3112 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.000115  3309 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:13:35.091754  3188 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.110571  3112 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.098461  3214 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:13:35.111063  3210 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:35.170718  3210 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:35.199841  3183 raft_consensus.cc:3055] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:35.200361  3183 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:35.201301  3183 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:35.203681   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.206220   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:35.207918   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:35.209178   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:35.213502  3188 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:35.223934  3298 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.224655  3298 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.227181  3280 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.248342  3247 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.249191  3247 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:35.251081  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.255070  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1456) took 1550 ms. Trace:
I20260709 13:13:35.255682  1003 rpcz_store.cc:276] 0709 13:13:33.704701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.704848 (+   147us) service_pool.cc:224] Handling call
0709 13:13:34.154734 (+449886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.158606 (+  3872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.254841 (+1096235us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54000483d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:35.254993 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001959168}
I20260709 13:13:35.262643  3258 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:35.264461   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.269550  3143 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:35.270167  3303 raft_consensus.cc:3055] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:35.270741  3303 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:35.270274  3143 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.290733   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1654) took 2005 ms. Trace:
I20260709 13:13:35.291296   846 rpcz_store.cc:276] 0709 13:13:33.285193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.288019 (+  2826us) service_pool.cc:224] Handling call
0709 13:13:33.288323 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.599926 (+311603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.290451 (+1690525us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54000fe954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:35.290641 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714889472}
I20260709 13:13:35.295022  3246 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.295747  3246 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.299947   923 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:35.302554   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1497) took 1761 ms. Trace:
I20260709 13:13:35.303097   922 rpcz_store.cc:276] 0709 13:13:33.540883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.541038 (+   155us) service_pool.cc:224] Handling call
0709 13:13:33.541477 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.076786 (+535309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.302226 (+1225440us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:35.302455 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229154560}
I20260709 13:13:35.305166  3303 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.302572  3175 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:13:35.314400  3305 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:35.314945  3305 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:35.321774   846 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:35.333225  3175 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:35.334076  3305 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.335482  3305 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:35.271453 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:35.412792  3183 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:35.415895   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 402708us
W20260709 13:13:35.417502   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.419559   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:35.430227   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:35.417179   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:35.432153   848 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:35.455370  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.459332   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:35.476504  1593 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.008s	sys 0.010s
W20260709 13:13:35.477986  1593 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.008s	sys 0.010s
I20260709 13:13:35.488497   846 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:35.490938   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 490052us
W20260709 13:13:35.493462   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 508107us
W20260709 13:13:35.494907   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.496373   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 230) took 10091 ms. Trace:
I20260709 13:13:35.504510  1001 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:13:35.504920  1001 pending_rounds.cc:85] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 5
I20260709 13:13:35.505127  1001 pending_rounds.cc:107] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:13:35.505354  1001 raft_consensus.cc:2889] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:35.509071  3331 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:35.509109   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78986.2us
I20260709 13:13:35.510952  3331 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.511505   896 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:35.428202  3303 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.523404   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:35.524478   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:35.524924  1601 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.008s
I20260709 13:13:35.525296  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:35.525251   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:35.490159  3169 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.496747   750 rpcz_store.cc:276] 0709 13:13:25.420552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.426367 (+  5815us) service_pool.cc:224] Handling call
0709 13:13:35.496283 (+10069916us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:31.760420 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:32.027908 (+267488us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:32.027969 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:13:32.027991 (+    22us) tablet.cc:662] Decoding operations
0709 13:13:32.028935 (+   944us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:32.029156 (+   221us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:32.029190 (+    34us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:13:32.029945 (+   755us) tablet.cc:701] Row locks acquired
0709 13:13:32.029999 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:13:32.030209 (+   210us) write_op.cc:270] Start()
0709 13:13:32.030378 (+   169us) write_op.cc:276] Timestamp: P: 1783602812030131 usec, L: 0
0709 13:13:32.030402 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:13:32.136168 (+105766us) log.cc:844] Serialized 2464 byte log entry
0709 13:13:32.607742 (+471574us) op_driver.cc:464] REPLICATION: finished
0709 13:13:32.806950 (+199208us) write_op.cc:301] APPLY: starting
0709 13:13:32.807054 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:32.807246 (+   192us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:32.807385 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:32.860692 (+ 53307us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:32.863916 (+  3224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:13:32.863969 (+    53us) write_op.cc:312] APPLY: finished
0709 13:13:32.864233 (+   264us) log.cc:844] Serialized 53 byte log entry
0709 13:13:32.864994 (+   761us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:32.865099 (+   105us) write_op.cc:454] Released schema lock
0709 13:13:32.865142 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:32.865502 (+   360us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:32.865529 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":198888,"apply.run_cpu_time_us":3111,"apply.run_wall_time_us":58546,"num_ops":6,"prepare.queue_time_us":350622,"prepare.run_cpu_time_us":30587,"prepare.run_wall_time_us":632532,"raft.queue_time_us":130526,"raft.run_cpu_time_us":20155,"raft.run_wall_time_us":212221,"replication_time_us":549942,"thread_start_us":498261,"threads_started":4,"wal-append.queue_time_us":254824,"wal-append.run_cpu_time_us":6727,"wal-append.run_wall_time_us":1470805}]]}
I20260709 13:13:35.515437   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:35.461102  3096 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.536257   896 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:35.541376   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:35.541376   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:35.544749   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:35.545928   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:35.519524  3143 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:35.548207   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:35.534286  3169 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.536037  3088 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.558096  3228 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.558605  3228 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.558961  3228 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.560647  3233 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:35.562028  3233 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:13:35.537308  3255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.282s	user 0.015s	sys 0.023s
I20260709 13:13:35.563045   753 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:35.542809  2979 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:35.549479  3198 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:35.553308  3263 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.516752  3321 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:13:35.546571  3174 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:13:35.569473   753 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:13:35.577044  3332 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:13:35.580348  1598 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:35.566681   818 leader_election.cc:400] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:35.557686  3169 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:35.565795  3255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.015s	sys 0.024s
I20260709 13:13:35.549467  3034 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:35.587453  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:35.588346   753 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 4 to 5, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:35.564527  3298 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:35.552706  3194 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 13:13:35.612087  3192 consensus_meta.cc:287] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.004s
I20260709 13:13:35.594533  3248 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:13:35.616994   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1685) took 1170 ms. Trace:
W20260709 13:13:35.617288  3111 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.009s
I20260709 13:13:35.617786  3111 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.617401   850 rpcz_store.cc:276] 0709 13:13:34.446568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.447179 (+   611us) service_pool.cc:224] Handling call
0709 13:13:34.448780 (+  1601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:34.452716 (+  3936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:34.452845 (+   129us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:35.283213 (+830368us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b2c00000c88. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:13:35.304704 (+ 21491us) log.cc:844] Serialized 26 byte log entry
0709 13:13:35.475846 (+171142us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:35.475901 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.475986 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:35.616671 (+140685us) raft_consensus.cc:1685] finished
0709 13:13:35.616693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.616931 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1523,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":5831,"spinlock_wait_cycles":10801792,"thread_start_us":170721,"threads_started":1,"wal-append.queue_time_us":170394}
I20260709 13:13:35.619673   930 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:35.541692  3322 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:13:35.565894  3234 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:13:35.565431  3182 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 13:13:35.593129  2959 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.006s
I20260709 13:13:35.605609  3144 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:35.615824  3298 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:35.604751   753 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 3, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:13:35.623538  3187 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.012s
I20260709 13:13:35.590696  3246 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:35.638902  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.647049   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:35.649974   850 raft_consensus.cc:606] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:35.650385   850 raft_consensus.cc:614] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:35.646139   753 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 5, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:35.630616  3211 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.652143   753 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:35.593550   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:35.654433   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60980.8us
I20260709 13:13:35.637207  3034 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:35.655325  3247 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:35.659684  3182 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:35.651177  3298 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:35.660313  3182 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:35.660710  3298 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.661268  3182 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.661691  3211 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:35.651623  3272 consensus_meta.cc:287] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.749s	user 0.004s	sys 0.020s
I20260709 13:13:35.664156  1002 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:35.650315  3181 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.743s	user 0.004s	sys 0.013s
W20260709 13:13:35.666903  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.664273   998 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:35.680843   929 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:35.600557  3112 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.644335  3255 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.685539   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:35.688684   753 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:35.782889  3206 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:35.784667  3034 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.786165  3256 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.786501  3181 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.787416  2959 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.791181  3112 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:35.707931  3194 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 13:13:35.707844  3321 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
W20260709 13:13:35.867147  3210 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.007s
W20260709 13:13:35.870699  3280 consensus_meta.cc:287] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.002s
I20260709 13:13:35.670943  3322 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
W20260709 13:13:35.808024   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 390863us
W20260709 13:13:35.808449   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:35.807583   996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.006s	sys 0.015s
I20260709 13:13:35.909073   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1468) took 1137 ms. Trace:
I20260709 13:13:35.909446   996 rpcz_store.cc:276] 0709 13:13:34.771291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.242881 (+471590us) service_pool.cc:224] Handling call
0709 13:13:35.243243 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:35.252457 (+  9214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.252505 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:35.253170 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:13:35.908719 (+655549us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:35.908763 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.908833 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:35.908873 (+    40us) raft_consensus.cc:1685] finished
0709 13:13:35.908885 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.909016 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8748,"raft.queue_time_us":11114,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":655044,"threads_started":1,"wal-append.queue_time_us":643962}
I20260709 13:13:35.719079  3255 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.701263  3111 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:35.824538   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 139071us
I20260709 13:13:35.826402   753 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:13:35.837169   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 292429us
I20260709 13:13:35.837991  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1588) took 1384 ms. Trace:
I20260709 13:13:35.919974  1002 rpcz_store.cc:276] 0709 13:13:34.453605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.662913 (+209308us) service_pool.cc:224] Handling call
0709 13:13:35.837810 (+1174897us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b54003e2e54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:35.837938 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232570240}
I20260709 13:13:35.838759  1000 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:35.920742   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:35.875639  3256 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:35.907841  1595 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.934s	user 0.015s	sys 0.003s
I20260709 13:13:35.825410  3206 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:35.859812  2959 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.885488  3098 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:13:35.890272   998 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1563) took 2686 ms (client timeout 1842 ms). Trace:
I20260709 13:13:35.840366  3263 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:35.892292   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35b46c75a1e049dd910275709bd62839"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:35.894608  3211 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:35.903697  3300 consensus_meta.cc:287] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.007s
W20260709 13:13:35.906410   999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.026s	sys 0.030s
I20260709 13:13:35.912870   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:35.818152   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 276814us
W20260709 13:13:35.916164   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 358420us
I20260709 13:13:35.920387   753 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 4, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:35.947384   753 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 4, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:35.948859  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:35.949501  1601 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:35.870970  3305 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.535s	user 0.003s	sys 0.004s
W20260709 13:13:35.879966  3258 consensus_meta.cc:287] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.617s	user 0.015s	sys 0.000s
I20260709 13:13:35.928956  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1601) took 1250 ms. Trace:
I20260709 13:13:35.905910  3210 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.940447   850 raft_consensus.cc:606] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:35.935842   998 rpcz_store.cc:269] 0709 13:13:33.203732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.212290 (+  8558us) service_pool.cc:224] Handling call
0709 13:13:35.890042 (+2677752us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b5400359b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:35.890182 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5585461504}
W20260709 13:13:35.941211   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 405049us
I20260709 13:13:35.923163  3255 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.959244  1000 rpcz_store.cc:276] 0709 13:13:34.678203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.918178 (+239975us) service_pool.cc:224] Handling call
0709 13:13:35.838408 (+920230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.920966 (+ 82558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.928714 (+  7748us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003e2e50. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:35.928886 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115582720}
I20260709 13:13:35.961521   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:35.973699  1601 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:35.962468   753 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:35.965874   995 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:35.974951  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:35.935298  3098 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:35.932773  3272 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:35.932452  1595 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:35.936142  3088 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:35.976984   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:35.991267  3320 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.991575  2959 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:35.977088   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64296.6us
I20260709 13:13:35.991768   930 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:35.991837  3320 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.993979   929 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:35.995565  3320 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:35.972668  3250 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.996781  3250 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:35.955231  3300 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:35.999572   921 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:35.999946  3300 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:36.000869  1000 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:36.002388  3098 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.004456  3331 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:36.007030  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:36.007375  1598 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:36.007845  1592 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
W20260709 13:13:36.011940  3215 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 1.028s	user 0.010s	sys 0.009s
I20260709 13:13:36.012368  3215 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.015868   819 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:36.016258  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1602) took 1208 ms. Trace:
I20260709 13:13:35.976027   753 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 3 to 7, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:35.973953   997 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:35.984020  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:35.985618   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:35.931735  3247 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:13:35.968029  3263 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:35.967144  3248 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
W20260709 13:13:36.011379   893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.002s	sys 0.044s
W20260709 13:13:36.067327   893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.002s	sys 0.044s
I20260709 13:13:35.988850   925 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:36.016682  1003 rpcz_store.cc:276] 0709 13:13:34.807428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.256960 (+449532us) service_pool.cc:224] Handling call
0709 13:13:35.257413 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:35.257882 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.257925 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:35.459091 (+201166us) log.cc:844] Serialized 26 byte log entry
0709 13:13:35.745804 (+286713us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:35.745866 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.745969 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.016019 (+270050us) raft_consensus.cc:1685] finished
0709 13:13:36.016038 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.016200 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13697,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":216053,"threads_started":1,"wal-append.queue_time_us":402264}
W20260709 13:13:36.037289  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.031344  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.029452   855 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
I20260709 13:13:36.024454   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1465) took 1255 ms. Trace:
W20260709 13:13:36.061878   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 76334.3us
I20260709 13:13:36.065765   753 catalog_manager.cc:5697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:36.076431   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:36.077077   925 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:36.063834   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:36.078126   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:36.078763   925 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:36.081640   997 raft_consensus.cc:2393] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 2: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:35.967547  3214 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:13:35.983768  3258 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.087175  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:36.063498  3348 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:13:36.017717  3337 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:13:36.018379  3343 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:36.028999  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 458) took 1103 ms. Trace:
I20260709 13:13:36.091536  1001 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.988036  3298 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:36.093425  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:36.016891  3322 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:36.003533  3098 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.104897   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:36.069178  3272 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:36.096982   925 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:36.067406  3320 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:35.989890  3342 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.029552  3215 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:36.095373   855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1482) took 2088 ms (client timeout 1621 ms). Trace:
I20260709 13:13:36.080219  3198 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.016708  3233 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 13:13:36.122964   855 rpcz_store.cc:269] 0709 13:13:34.001686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.007683 (+  5997us) service_pool.cc:224] Handling call
0709 13:13:36.089596 (+2081913us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54001293d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:36.089712 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4614999424}
I20260709 13:13:36.114128   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:36.083696  3300 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.125130   818 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:36.129297   925 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:36.018573  3228 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.083756  3088 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:13:36.107761  3322 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.084928  3331 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:13:36.090121  3096 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:13:36.176599  3250 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:36.107828  3181 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.178514  3181 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:36.179205  3352 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:36.090373  2959 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:13:36.185464  3336 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:13:36.187018  2959 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.187798  2959 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.082056   999 rpcz_store.cc:276] 0709 13:13:34.769514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.949237 (+179723us) service_pool.cc:224] Handling call
0709 13:13:34.949505 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.949697 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:35.339925 (+390228us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.023947 (+684022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:36.024038 (+    91us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:36.024111 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.024216 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.024373 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80438,"prepare.queue_time_us":137797,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":85842,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":12915,"thread_start_us":969672,"threads_started":2,"wal-append.queue_time_us":597148}
I20260709 13:13:36.192147  2959 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:36.100459  3246 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:13:36.176452  3215 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:13:36.148671   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.108572  2979 consensus_queue.cc:1059] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:13:36.201505   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.204854  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.090117  3169 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.128667  3335 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.223958  3335 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.100031  1595 rpcz_store.cc:276] 0709 13:13:34.921791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.922928 (+  1137us) service_pool.cc:224] Handling call
0709 13:13:36.025322 (+1102394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:36.226644  3181 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:36.117740  3321 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:13:36.120657  3192 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:13:36.142762  3344 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.220907  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:36.217628  3183 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.218541  1002 raft_consensus.cc:2368] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:36.083895  3206 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.105976  3258 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.142448  3320 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:36.169639  3322 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.090600  3256 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.238052   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:36.213321  3338 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.217639  3211 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:13:36.239363   924 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:36.245983   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
W20260709 13:13:36.247493   895 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.621s (SENT)
I20260709 13:13:36.138255  3198 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.230218  3215 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:36.275215  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:36.230253  3329 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:13:36.227357  3335 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:36.283839   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:36.261818  3206 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:36.253798  3320 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.350813  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:36.295176   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:36.286252   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 334445us
I20260709 13:13:36.209263  3263 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.295025  3187 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.333832   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:36.337381  1001 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:36.131807  3342 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.281181  3111 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.296707  1594 raft_consensus.cc:2410] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:36.252800  3298 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:36.148594  3300 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.246737  3183 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.289565  3233 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.236497  3344 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.352520  1004 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:36.305802  3247 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:13:36.332113   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.392825   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.297545  3210 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:36.339134  3250 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:36.347306  3336 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:36.253518  3272 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.349918  3181 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:36.347394  3331 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:36.375618   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 128116us
I20260709 13:13:36.353988  3320 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.350354  3321 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.284163  3338 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.349608  3335 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:36.384054  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 326506us
I20260709 13:13:36.385110   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:36.385764   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:36.388514   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:36.390034   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:36.457163   846 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:36.406996   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:36.406464   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 325149us
I20260709 13:13:36.410845   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:36.420333  3334 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.410298  3250 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.392328  3247 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:36.369074  3246 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:13:36.402422  3363 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.383665  3298 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.392253  3215 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.427732  3331 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.393843  3341 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:13:36.374634  3111 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.429076  3344 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:36.429965  3342 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:36.365774  3187 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.435992  3210 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.427716  3361 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.437526  3280 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.439716   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:36.467253   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 230) took 1468 ms. Trace:
I20260709 13:13:36.508375  3192 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:13:36.472422  3335 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:36.417591  3228 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:36.398761  3233 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:36.524679   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 68757.5us
I20260709 13:13:36.525295  3198 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:36.595379   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 227489us
W20260709 13:13:36.485970  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.468837  3321 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.505440  3334 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.555972  3363 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:36.621423   852 rpcz_store.cc:276] 0709 13:13:34.999140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.999291 (+   151us) service_pool.cc:224] Handling call
0709 13:13:34.999655 (+   364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:35.496139 (+496484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.496170 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:35.496821 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 13:13:35.915933 (+419112us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:35.944509 (+ 28576us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.944779 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.944873 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.467025 (+522152us) raft_consensus.cc:1685] finished
0709 13:13:36.467051 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.467186 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4608,"prepare.queue_time_us":30208,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":92484,"raft.run_cpu_time_us":772,"raft.run_wall_time_us":100634,"thread_start_us":914741,"threads_started":2,"wal-append.queue_time_us":478088}
W20260709 13:13:37.005591   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.643165   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 184474us
I20260709 13:13:37.015271   849 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:37.018532   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1635) took 1030 ms. Trace:
I20260709 13:13:37.019482   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:36.997206   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.041209  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.037040   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:36.624625   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.055925   922 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7.
W20260709 13:13:37.061298   970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.033s
W20260709 13:13:37.061823   970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.000s	sys 0.034s
I20260709 13:13:37.022082   846 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:37.019019   926 rpcz_store.cc:276] 0709 13:13:35.988435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.989167 (+   732us) service_pool.cc:224] Handling call
0709 13:13:35.989462 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.015203 (+ 25741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.022676 (+  7473us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.023190 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.353559 (+330369us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:36.355502 (+  1943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.355589 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.018262 (+662673us) raft_consensus.cc:1685] finished
0709 13:13:37.018285 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.018465 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91401,"raft.queue_time_us":28019,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":55579,"thread_start_us":330020,"threads_started":1,"wal-append.queue_time_us":403764}
I20260709 13:13:37.021085   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:37.064302   849 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:36.642529  3088 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.411s
W20260709 13:13:37.056378   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 616705us
W20260709 13:13:37.055948   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.058681   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50482us
I20260709 13:13:37.082625   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:36.663215  3355 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:13:36.795512  3206 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.066648   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.092273   926 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:37.073603   847 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.001s	sys 0.009s
W20260709 13:13:37.093797   847 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.001s	sys 0.009s
W20260709 13:13:36.740234  3353 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
W20260709 13:13:37.098086   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.624734  3258 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.052369  3248 raft_consensus.cc:3055] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:36.934916  3214 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:37.053979  3307 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.620545  3280 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.019133  3309 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:36.724287  3344 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:37.105371   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:36.624362  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.110517  3321 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.122329   847 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:37.124848   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:37.125488   851 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.119244   926 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:36.624608  3210 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:37.125484   925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.037s
I20260709 13:13:37.141016   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.141870   926 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:37.090363  3198 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.142843   925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.012s	sys 0.037s
I20260709 13:13:36.624516  3361 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:37.126752  3144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.031s	sys 0.031s
I20260709 13:13:37.074848  3272 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:13:37.173871   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.182123   924 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:37.105589  3206 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.104873  3248 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:37.112023  3258 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.112943  3098 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.130740  3181 consensus_meta.cc:287] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.003s
I20260709 13:13:37.105484  3187 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:37.123092  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:37.145004  3341 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.006s	sys 0.017s
I20260709 13:13:37.215570  3381 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.136459  3034 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.216073  3363 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.216228  3381 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.216301  3034 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.218622  3034 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:37.218627  3381 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.215508  1592 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.219388  3381 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.220284  3358 log.cc:927] Time spent T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.330s	user 0.001s	sys 0.001s
I20260709 13:13:37.219969  3381 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.221442   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1630) took 1238 ms. Trace:
I20260709 13:13:37.222630  3272 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:37.147595  3250 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.223258  3394 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:36.624841  3247 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.223861  3394 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:37.138540  3371 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:37.224455  3365 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:13:37.152005  3111 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.224802  3371 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.224836  3394 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.225558   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1637) took 1236 ms. Trace:
I20260709 13:13:37.226496  3394 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:37.225948   929 rpcz_store.cc:276] 0709 13:13:35.988921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.994770 (+  5849us) service_pool.cc:224] Handling call
0709 13:13:35.995090 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.080575 (+ 85485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.080608 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.081072 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.259126 (+178054us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:36.259163 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.259230 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.225332 (+966102us) raft_consensus.cc:1685] finished
0709 13:13:37.225355 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.225496 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91946,"raft.queue_time_us":111086,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8763,"thread_start_us":142527,"threads_started":1,"wal-append.queue_time_us":866738}
I20260709 13:13:37.125453  3307 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.205036   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.228238   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.228582   924 raft_consensus.cc:2376] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:13:37.125980  3214 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:37.154683   821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.008s	sys 0.028s
I20260709 13:13:37.162376  3382 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.208289  3096 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:37.154204  3198 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:37.215817  3341 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.006s	sys 0.017s
I20260709 13:13:37.157652  3321 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.223857  3250 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:37.179291  3144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.033s	sys 0.037s
I20260709 13:13:37.130627  3344 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.221874   923 rpcz_store.cc:276] 0709 13:13:35.984030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.984170 (+   140us) service_pool.cc:224] Handling call
0709 13:13:35.984533 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.015508 (+ 30975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.015535 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.016005 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.379877 (+363872us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:36.379927 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.448576 (+ 68649us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.221177 (+772601us) raft_consensus.cc:1685] finished
0709 13:13:37.221213 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.221376 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104400,"raft.queue_time_us":37115,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":57174,"thread_start_us":363513,"threads_started":1,"wal-append.queue_time_us":626517}
I20260709 13:13:37.229737   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.224466  3247 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:37.100196  3183 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.230731  3214 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.130774  3309 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.231267   929 raft_consensus.cc:2410] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:13:37.231621  3377 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.390525  3382 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.408979  3377 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.409394   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.410012   929 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:37.410199  3309 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.410449  3382 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.411155  3377 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.227280  3371 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.411058  3250 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.235831  3361 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.236465  3381 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:37.236471  3307 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.210152  3335 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.243103   851 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:37.407272  3321 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.247014  1592 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:37.416833   929 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:37.417569  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.417898  3321 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.418067   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.418247  1592 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:37.418586   929 raft_consensus.cc:2376] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:13:37.208462  3258 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.419921   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.250306  3206 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.251868  3280 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:37.255201  3368 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
I20260709 13:13:37.257488  3210 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:37.262888  3322 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.981s	user 0.006s	sys 0.003s
I20260709 13:13:37.424497  3322 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.416836   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:37.268332  3248 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.164472  3372 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:37.269959  3334 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.419384  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.199379  3387 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.205727  3331 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:37.273250   848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.014s	sys 0.006s
I20260709 13:13:37.180624  3338 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:37.274464   928 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.000s	sys 0.021s
W20260709 13:13:37.431591   928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.000s	sys 0.021s
I20260709 13:13:37.431092  3387 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.223189  3272 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.271435  3181 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:37.272423  3363 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.274806   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1628) took 1294 ms. Trace:
W20260709 13:13:37.292374  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.295820  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.436380  1598 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:37.437390  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.436051   927 rpcz_store.cc:276] 0709 13:13:35.980455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.981900 (+  1445us) service_pool.cc:224] Handling call
0709 13:13:35.982212 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.982464 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:36.344305 (+361841us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.274370 (+930065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:37.274444 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:37.274504 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.274591 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.274741 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":241641,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":343135,"spinlock_wait_cycles":14829440,"thread_start_us":742143,"threads_started":3,"wal-append.queue_time_us":887785}
I20260709 13:13:37.438237  1598 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:37.274276  3111 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.213855  3098 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.311307   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.313185  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.313606  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.315017  3187 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:37.440966  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:37.280026  3386 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.441577  1600 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:37.439558  3363 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.344399   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:37.344812   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.430696  3372 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:37.448365   998 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:13:37.448788   998 pending_rounds.cc:85] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 3
I20260709 13:13:37.345693   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:37.449018   998 pending_rounds.cc:107] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:13:37.449141  3187 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.449299   852 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:37.449792   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:37.449669  3187 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:37.450259   967 leader_election.cc:336] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.554s (SENT)
I20260709 13:13:37.450805   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:37.344404   930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.001s	sys 0.008s
I20260709 13:13:37.330536  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:37.460175   995 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:13:37.460620   995 pending_rounds.cc:85] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 6
I20260709 13:13:37.460850   995 pending_rounds.cc:107] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:13:37.461077   995 raft_consensus.cc:2889] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:37.357825  3398 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.462255   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.358832  3233 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.432974  1592 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:37.365824  3401 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.365689  3305 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:37.370397  3351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.007s
I20260709 13:13:37.370967   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:37.485306   855 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:37.377784   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:37.380357  3320 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.914s	user 0.016s	sys 0.000s
I20260709 13:13:37.383627  3390 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:37.387037   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1643) took 1149 ms. Trace:
I20260709 13:13:37.496685   925 rpcz_store.cc:276] 0709 13:13:36.237987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.366627 (+128640us) service_pool.cc:224] Handling call
0709 13:13:36.381951 (+ 15324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.394371 (+ 12420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.394413 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.403344 (+  8931us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.246124 (+842780us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:37.246168 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.246243 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.386817 (+140574us) raft_consensus.cc:1685] finished
0709 13:13:37.386838 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.386973 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11963,"raft.queue_time_us":36407,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":692510,"threads_started":1,"wal-append.queue_time_us":880421}
W20260709 13:13:37.388779   821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.010s	sys 0.031s
W20260709 13:13:37.391628  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.499387   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 1.10921e+06us
I20260709 13:13:37.395275  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.394402  3397 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.406780   846 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:37.505052   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1633) took 1518 ms. Trace:
I20260709 13:13:37.452862  3387 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.405460  3198 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.408277   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.389664  3096 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:37.489787  3351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.000s	sys 0.008s
I20260709 13:13:37.421424   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.518553   929 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:37.423954  3258 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.425712   854 raft_consensus.cc:2368] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
W20260709 13:13:37.266407  3255 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.283s	user 0.010s	sys 0.000s
I20260709 13:13:37.426128   851 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:37.270339  3298 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.524048  3096 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.423293   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:37.525295  3096 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:37.429723   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.430881   848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.014s	sys 0.007s
I20260709 13:13:37.412892  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 303) took 1135 ms. Trace:
I20260709 13:13:37.432760  3344 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.429224  3322 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.440611  3111 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.316112   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.345638  3300 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.546213  3351 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.442643   922 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:37.441295  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.443022   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.548733   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1531) took 1109 ms. Trace:
I20260709 13:13:37.549181   927 rpcz_store.cc:276] 0709 13:13:36.439213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.442785 (+1003572us) service_pool.cc:224] Handling call
0709 13:13:37.548660 (+105875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:37.441694  3386 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.445389  3272 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:37.445537  3363 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.446062  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.446511  3371 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:37.447142  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.442286  3307 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:37.451751  3187 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:37.451995   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.578s (SENT) [suppressed 67 similar messages]
I20260709 13:13:37.452378  3341 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:13:37.452852   930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.001s	sys 0.008s
I20260709 13:13:37.594491   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1636) took 1605 ms. Trace:
I20260709 13:13:37.594873   930 rpcz_store.cc:276] 0709 13:13:35.988693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.992834 (+  4141us) service_pool.cc:224] Handling call
0709 13:13:35.993139 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.487896 (+494757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.487941 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.556494 (+ 68553us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.594166 (+1037672us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:37.594204 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.594270 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.594306 (+    36us) raft_consensus.cc:1685] finished
0709 13:13:37.594316 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.594434 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38703,"raft.queue_time_us":99768,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":11998,"thread_start_us":1398394,"threads_started":2,"wal-append.queue_time_us":792814}
I20260709 13:13:37.448382  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:37.596535  1001 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:37.556272   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.449318   998 raft_consensus.cc:2889] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:37.457432   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:37.598213   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 140810us
I20260709 13:13:37.452179  3372 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.356508  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:37.456641  1566 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.360917  3256 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:37.471954  1003 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:37.439702  3399 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.474879   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.486438   849 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
W20260709 13:13:37.499974  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.473579  3398 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.508973   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:37.513790  1600 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:37.514463   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1521) took 1085 ms. Trace:
I20260709 13:13:37.415215  3250 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.505616   928 rpcz_store.cc:276] 0709 13:13:35.987088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.987196 (+   108us) service_pool.cc:224] Handling call
0709 13:13:35.987496 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:36.014802 (+ 27306us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:36.353298 (+338496us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.431925 (+1078627us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:37.432525 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 13:13:37.432705 (+   180us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:37.432751 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.432820 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.504760 (+ 71940us) raft_consensus.cc:1685] finished
0709 13:13:37.504788 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.504975 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167002,"prepare.queue_time_us":209657,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":370354,"raft.run_cpu_time_us":4509,"raft.run_wall_time_us":210070,"thread_start_us":1267887,"threads_started":3,"wal-append.queue_time_us":778357}
I20260709 13:13:37.522992   855 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:37.484488  3181 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:37.534888   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 111645us
I20260709 13:13:37.542447   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:37.543828   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 227787us
I20260709 13:13:37.504228  3397 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.543294  3334 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:13:37.511991  3233 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:37.536053  3255 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.640031   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.514756  3390 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:37.496380  3305 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.506311  3320 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.546892  3351 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.489499  3401 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.698195   922 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:37.628445   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.709893   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.635275   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:37.752153   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 243267us
W20260709 13:13:37.628948   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.753181   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.659794  1001 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:37.753510   995 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:37.609622  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 153015us
I20260709 13:13:37.815423   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:37.649653   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:37.588888  1002 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:37.650962   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1509) took 1265 ms. Trace:
I20260709 13:13:37.750607   969 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.754962   923 rpcz_store.cc:276] 0709 13:13:36.429541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.407993 (+978452us) service_pool.cc:224] Handling call
0709 13:13:37.514404 (+106411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:37.589386  3387 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:37.616691   927 raft_consensus.cc:2410] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:37.758486   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:37.758510   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 156373us
I20260709 13:13:37.640826  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.588294  3338 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.758112  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:37.851994   922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1520) took 1422 ms (client timeout 1849 ms). Trace:
I20260709 13:13:37.540858  3298 raft_consensus.cc:2764] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:37.903618   921 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:37.593477  3112 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:37.902864   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.904057   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.905784   926 raft_consensus.cc:2376] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4. Re-sending same reply.
W20260709 13:13:37.906306   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 155731us
I20260709 13:13:37.637652  3280 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:37.590786  3210 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:37.703804  3215 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.880681   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 428644us
I20260709 13:13:37.911055  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.906404   928 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:37.851585   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.909528   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:37.913057   922 rpcz_store.cc:269] 0709 13:13:36.429282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.344136 (+914854us) service_pool.cc:224] Handling call
0709 13:13:37.851930 (+507794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:37.910133  1004 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:37.906744  1003 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:37.852662  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.610273  3307 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.801180   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1511) took 1415 ms. Trace:
I20260709 13:13:37.642019  3174 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:37.943145  1000 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
W20260709 13:13:37.806629  3420 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.261s	user 0.007s	sys 0.004s
I20260709 13:13:37.713194  3386 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.623008  3258 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.320s
I20260709 13:13:37.942034   924 raft_consensus.cc:2376] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6. Re-sending same reply.
I20260709 13:13:37.909242   854 rpcz_store.cc:276] 0709 13:13:36.385108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.385221 (+   113us) service_pool.cc:224] Handling call
0709 13:13:37.650792 (+1265571us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b540041f0d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:37.650909 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222783360}
I20260709 13:13:37.943051   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.806840  3331 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.941743   850 raft_consensus.cc:606] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:37.984321   850 raft_consensus.cc:614] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:37.984562   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:37.985342  1593 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:37.756970  3321 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:37.771330  3256 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.806480  3111 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:37.678562  3214 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.609961  3198 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.944180   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.916698   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.917207   930 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:37.966063   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.983361  3386 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:37.969349   995 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
W20260709 13:13:37.966359   967 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.672s (SENT)
I20260709 13:13:37.968824  1001 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:37.592976  3361 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:37.764011   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:37.945056  3338 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:37.966893   848 rpcz_store.cc:276] 0709 13:13:36.385678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.385802 (+   124us) service_pool.cc:224] Handling call
0709 13:13:36.386184 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.466316 (+ 80132us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:36.723443 (+257127us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.529308 (+805865us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:37.537089 (+  7781us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.537319 (+   230us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:37.537371 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.537466 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.800884 (+263418us) raft_consensus.cc:1685] finished
0709 13:13:37.800921 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.801115 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8847,"prepare.queue_time_us":256673,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":82695,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":52253,"thread_start_us":876213,"threads_started":2,"wal-append.queue_time_us":633182}
I20260709 13:13:37.806946  3300 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.689853  3396 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:37.978479   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.978644  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.625241  3098 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.015714  1003 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 2.
W20260709 13:13:37.640069  3417 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:13:37.594311  3371 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:37.915233  3096 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:37.943234   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.915546  3280 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.943959  3272 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.772768  3181 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:37.986754   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 427959us
I20260709 13:13:37.964671  3174 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:37.908166  3250 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:37.908676   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:37.965569  3387 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:37.986790   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78919.2us
I20260709 13:13:37.990382  3401 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:37.988165  3403 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:37.993988  3404 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:37.997062  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:37.980639  3183 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:37.942613  3112 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.071700  3280 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.001802   998 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.007436   996 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:37.975018  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 122415us
W20260709 13:13:38.008121   967 leader_election.cc:336] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.710s (SENT)
I20260709 13:13:37.973822  3210 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.100752  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:38.012426   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 107007us
I20260709 13:13:37.761396  3399 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.089054  3387 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:38.130568  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:38.012902   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 204432us
I20260709 13:13:37.972049  3215 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.014634   929 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:37.996340  3214 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.013037  3198 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:37.988292  3331 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:38.040624  3375 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.950s	user 0.003s	sys 0.000s
I20260709 13:13:37.535419  1595 rpcz_store.cc:276] 0709 13:13:36.273806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.288715 (+ 14909us) service_pool.cc:224] Handling call
0709 13:13:36.370206 (+ 81491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.437235 (+ 67029us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18000315a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:36.732502 (+295267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.757922 (+ 25420us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:37.121139 (+363217us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.362632 (+241493us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:37.364116 (+  1484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.384952 (+ 20836us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.389017 (+  4065us) raft_consensus.cc:1685] finished
0709 13:13:37.389964 (+   947us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.408685 (+ 18721us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10212,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":7618432,"thread_start_us":365166,"threads_started":2,"wal-append.queue_time_us":228554}
W20260709 13:13:38.046101  3422 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.334s	user 0.001s	sys 0.005s
I20260709 13:13:38.087775  3272 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:38.049626  3413 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.409s	user 0.000s	sys 0.008s
I20260709 13:13:38.050293  3142 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:38.129624  3403 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:38.146603  3381 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:38.054241  3307 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.054796  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.009022  3390 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.019147  3396 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.056542  3169 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.057406  3371 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.058948  3397 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.005911  3386 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.991407  3334 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:38.063079   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51502 (request call id 17) took 1864 ms. Trace:
W20260709 13:13:38.064523   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 155106us
W20260709 13:13:38.067821   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 82031.3us
I20260709 13:13:38.083475   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.090077   927 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:38.093436  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.095651   925 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.018164  3300 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.092108  1002 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:38.070576  3096 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.965199   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.097500  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.101389   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:38.114548  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.136387   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.143059   996 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:38.095964  3404 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.146979   928 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 2.
I20260709 13:13:38.095055  3183 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.048182  3256 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.097497  3398 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.011387  3361 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.144899  3401 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.136806  3320 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.093192  3250 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.186422  3386 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.146909  3429 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:38.153960  3372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.013s	sys 0.016s
I20260709 13:13:38.179780   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.286531   854 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:38.171819  3142 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:38.169250  3351 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.173215  3381 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.244451   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1543) took 1109 ms. Trace:
I20260709 13:13:38.243127   850 rpcz_store.cc:276] 0709 13:13:36.196462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.245830 (+ 49368us) service_pool.cc:224] Handling call
0709 13:13:38.060801 (+1814971us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001cc5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b39853 00007f0d07b0760a 00007f0d0108d7d4 00007f0d0108d762 00007f0d0108d691 00007f0d0108d60c 00007f0d0108c8c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:38.060912 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3813882112}
W20260709 13:13:38.246752   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 163363us
I20260709 13:13:38.247404  1003 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:38.249611   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1545) took 1038 ms. Trace:
I20260709 13:13:38.289244   927 rpcz_store.cc:276] 0709 13:13:37.135391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.908283 (+772892us) service_pool.cc:224] Handling call
0709 13:13:38.244404 (+336121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.182363  3423 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.179733  3307 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:38.281143  3436 log.cc:927] Time spent T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:13:38.252381  3215 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.253479  3112 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.254519  3183 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:38.255683  3413 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:13:38.255713  3272 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.256235   925 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:38.179733  3111 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.259230  3210 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.149154  3174 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.260926  3334 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:38.335966  3174 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:38.335963  3334 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.337539  3396 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.309124  3423 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.338265  3334 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:38.340764   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.261615  3248 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:38.263871   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.266126   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.342445   930 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:38.342902   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 268) took 1021 ms. Trace:
I20260709 13:13:38.343194   930 rpcz_store.cc:276] 0709 13:13:37.321291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.265832 (+944541us) service_pool.cc:224] Handling call
0709 13:13:38.342847 (+ 77015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.344669   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:38.345744   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.346534   930 raft_consensus.cc:2410] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:38.347240   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.273087  3399 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.347769   929 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:38.276928  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:38.348606  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71794.3us
I20260709 13:13:38.349324   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.350004   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1526) took 1919 ms. Trace:
I20260709 13:13:38.350288   929 rpcz_store.cc:276] 0709 13:13:36.430812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.349118 (+1918306us) service_pool.cc:224] Handling call
0709 13:13:38.349956 (+   838us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:38.053996   924 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1532) took 1580 ms (client timeout 1774 ms). Trace:
W20260709 13:13:38.280788   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 272699us
W20260709 13:13:38.351776   924 rpcz_store.cc:269] 0709 13:13:36.473148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.229457 (+756309us) service_pool.cc:224] Handling call
0709 13:13:38.053752 (+824295us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54004461d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:38.053914 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227960192}
I20260709 13:13:38.352875   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.353549   924 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:38.353603   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:38.281541   821 leader_election.cc:400] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:38.354354   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.354743   821 leader_election.cc:403] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:13:38.354923   924 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:38.355732   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 74325.1us
I20260709 13:13:38.355986  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.291999   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.357365  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.357524  3431 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:38.181946  3255 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.359745  3399 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:38.360152  3399 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.360497  3399 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.286072  3401 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.361997   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:38.363183  3401 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.293453  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:38.363500  3255 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:38.363897  1002 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.293918  3142 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.284633  3371 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.368474  3371 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.368629   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:38.369144  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.370106  3255 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:38.370496  3371 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.294796  3307 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.371559  1600 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:38.372643  3307 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.373070  1593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 468) took 1058 ms. Trace:
I20260709 13:13:38.373430  1593 rpcz_store.cc:276] 0709 13:13:37.315072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.394982 (+ 79910us) service_pool.cc:224] Handling call
0709 13:13:38.372879 (+977897us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400270954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:38.373016 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113436928}
I20260709 13:13:38.374495   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1638) took 2385 ms. Trace:
I20260709 13:13:38.374936   921 rpcz_store.cc:276] 0709 13:13:35.989161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.001626 (+ 12465us) service_pool.cc:224] Handling call
0709 13:13:36.001947 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:37.904744 (+1902797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.374185 (+469441us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b540031b5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:38.374427 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4372675968}
I20260709 13:13:38.376365   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.252125  3214 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:38.287819  3372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.013s	sys 0.016s
I20260709 13:13:38.248654  3404 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.368674  3142 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.378880   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:38.295359   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49768.1us
I20260709 13:13:38.298831   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.298935   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.299247  3433 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:38.299862  3381 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.380187   998 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:38.379580   921 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 2.
I20260709 13:13:38.380697  3305 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.381548   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.382711   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.289815   922 rpcz_store.cc:276] 0709 13:13:37.211063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.964916 (+753853us) service_pool.cc:224] Handling call
0709 13:13:38.249538 (+284622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.300123  3181 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.301995   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 239) took 1006 ms. Trace:
I20260709 13:13:38.383761   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.383920   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.384382   921 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:38.385860   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.350304  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.289038  3429 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.387946  1599 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.386598   921 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 2.
I20260709 13:13:38.388609   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1542) took 1268 ms. Trace:
I20260709 13:13:38.355378  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.388903   921 rpcz_store.cc:276] 0709 13:13:37.120104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.385644 (+1265540us) service_pool.cc:224] Handling call
0709 13:13:38.388562 (+  2918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.384079   853 rpcz_store.cc:276] 0709 13:13:37.296002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.450639 (+154637us) service_pool.cc:224] Handling call
0709 13:13:37.457814 (+  7175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:37.903693 (+445879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:37.903723 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:37.903773 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:37.903807 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.301743 (+397936us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400071554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:38.301791 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.301935 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024926976}
I20260709 13:13:38.389658  3429 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.302351  3403 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.389902  3300 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:38.284838  3382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.017s	sys 0.035s
I20260709 13:13:38.275796  3359 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.309597   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.289398  3338 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.316138   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:38.323978   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.324836  3361 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.320250  3309 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:13:38.330859  3387 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.331707  3111 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.332808  3428 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:38.334501  3280 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.343780  3215 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:38.247088  3096 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.359113  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 449) took 1340 ms. Trace:
I20260709 13:13:38.292701  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.432324  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:38.432507   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.432561  1001 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.433014  1595 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.433547  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.384582   922 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:38.435557   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.436932   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.437613   922 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:38.438591   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:38.304275   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:38.304354  3250 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.389870   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.340060  3423 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.442780   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.442870   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.395576   923 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:38.395989   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.397718   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.400759  1002 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:38.404075   847 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:38.405093   851 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:38.444715   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.445520   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.407318   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:38.394563  3382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.017s	sys 0.035s
I20260709 13:13:38.412549   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:38.451555   929 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.412853  3272 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:38.414691   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.347769   930 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:38.422838  3248 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:38.417802   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.417698  3183 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.351451  1592 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:38.420131   894 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:38.469612   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49594.7us
I20260709 13:13:38.470590   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.420537   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:38.421741   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 105684us
W20260709 13:13:38.471455   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73791.2us
I20260709 13:13:38.472622   819 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.472730   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.402850   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.421792   855 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:38.474604   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.474831   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.475769  3194 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.475833   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.476341  3194 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.422940  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.477559  1003 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:38.419049  3399 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.431382  1601 rpcz_store.cc:276] 0709 13:13:37.019094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.020119 (+  1025us) service_pool.cc:224] Handling call
0709 13:13:37.029179 (+  9060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:37.413477 (+384298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:37.413505 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:37.413549 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:37.413582 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.358895 (+945313us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54007102d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:38.358942 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.359060 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883305728}
I20260709 13:13:38.439321   922 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:38.481900   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 263) took 2125 ms. Trace:
W20260709 13:13:38.422559  3418 log.cc:927] Time spent T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.779s	user 0.004s	sys 0.011s
I20260709 13:13:38.395292   927 raft_consensus.cc:2410] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:38.447767  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.448470  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:38.452075   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 56186.9us
W20260709 13:13:38.448690   738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 144448us
I20260709 13:13:38.417146  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.428812  3401 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:13:38.462641   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.463981   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.457319   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1551) took 1003 ms. Trace:
I20260709 13:13:38.465742  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.468243   996 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:38.474529  3364 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:13:38.429669   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.479405   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.480207   846 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:38.482255   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 264) took 2121 ms. Trace:
I20260709 13:13:38.429294  3228 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 13:13:38.561216  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.656889  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:38.487896   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:38.656873   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 114359us
I20260709 13:13:38.659516   924 rpcz_store.cc:276] 0709 13:13:37.453827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.355569 (+901742us) service_pool.cc:224] Handling call
0709 13:13:38.457263 (+101694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.660766   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.661921   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.662995   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.663806   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.558633   929 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:38.482266   930 rpcz_store.cc:276] 0709 13:13:36.356367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.479083 (+2122716us) service_pool.cc:224] Handling call
0709 13:13:38.481483 (+  2400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.481636 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.481657 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.481694 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:38.481726 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.481774 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.481856 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.666318  1000 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:38.667022   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1546) took 1450 ms. Trace:
I20260709 13:13:38.667505   927 rpcz_store.cc:276] 0709 13:13:37.216811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.309326 (+1092515us) service_pool.cc:224] Handling call
0709 13:13:38.666964 (+357638us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:38.667946   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 195305us
W20260709 13:13:38.668067   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 195491us
I20260709 13:13:38.465267  3188 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.657011   922 rpcz_store.cc:276] 0709 13:13:36.360813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.481617 (+2120804us) service_pool.cc:224] Handling call
0709 13:13:38.481845 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.481994 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.482017 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.482054 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:38.482081 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.482128 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.482211 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.709172   847 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:38.672463  1004 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
W20260709 13:13:38.709839  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 391346us
I20260709 13:13:38.472622   894 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:38.710798   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 238215us
I20260709 13:13:38.676317   853 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:38.711133  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:38.712982  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:38.713984   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45037.5us
I20260709 13:13:38.714831   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:38.716439   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 47217.5us
I20260709 13:13:38.715876   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.714387  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:38.480592  3423 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.475437  3377 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:38.685266  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.786784  1001 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:38.478760  3272 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:38.502507  3429 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:38.443063  3247 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.456748  3142 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.710158   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1527) took 2279 ms. Trace:
I20260709 13:13:38.720058  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:38.770622  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56321.1us
I20260709 13:13:38.772009   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.431476  3428 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:38.717816   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 54888.8us
I20260709 13:13:38.819949   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.773525   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.769822  3361 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.725677   930 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:38.832836   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1525) took 2402 ms. Trace:
I20260709 13:13:38.720180   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.834304   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.835312   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.836166  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.792745  3344 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.480608  3338 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:38.682574   848 raft_consensus.cc:2368] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
I20260709 13:13:38.842216   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.792155   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.793913  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.449440  3359 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.707147  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.593569  3280 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.442364  3215 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.666906  3433 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:38.586930  3111 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.499321  3194 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:38.713516  3258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.019s	sys 0.032s
I20260709 13:13:38.684929  3174 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.810200   852 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:38.857642  3443 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:13:38.802944   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.872932   924 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:38.800369   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.810714   922 rpcz_store.cc:276] 0709 13:13:36.431048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.709382 (+2278334us) service_pool.cc:224] Handling call
0709 13:13:38.709673 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.709861 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.709885 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.709923 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:38.709956 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.710012 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.710110 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:38.812057   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 96240us
I20260709 13:13:38.813251  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.850293  3399 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.876297  1601 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:38.814540  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:38.877703  1004 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:38.815939  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.595746  3387 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.597980  3305 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:38.717864  3187 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.718163  3309 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.817926  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:38.884799   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:38.717795  3307 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:38.863749  3097 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.499339  1597 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:38.768321  3250 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:38.770212  3263 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.729317  3320 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:38.768509  3341 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:38.467732  3248 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.833207   930 rpcz_store.cc:276] 0709 13:13:36.430558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.697325 (+2266767us) service_pool.cc:224] Handling call
0709 13:13:38.725510 (+ 28185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.832590 (+107080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.832782 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.839593   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.840705   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1616) took 1393 ms. Trace:
I20260709 13:13:38.898680   997 rpcz_store.cc:276] 0709 13:13:37.446797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.448487 (+  1690us) service_pool.cc:224] Handling call
0709 13:13:37.448777 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:37.454769 (+  5992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:37.454791 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:37.455128 (+   337us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.489518 (+ 34390us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:37.489567 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.489641 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:38.362725 (+873084us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:38.817274 (+454549us) raft_consensus.cc:1685] finished
0709 13:13:38.817294 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.840642 (+ 23348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5632,"raft.queue_time_us":5740,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":9995,"spinlock_wait_cycles":3685120,"thread_start_us":29203,"threads_started":1,"wal-append.queue_time_us":130606}
W20260709 13:13:38.845252   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 53218.4us
I20260709 13:13:38.900873   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.901968   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.770036  3192 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:13:38.848595   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:38.848831  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:38.851435  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:38.829141  3382 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.853471   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.828737  3356 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.851037  3423 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.914667   924 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:38.915467   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1528) took 2484 ms. Trace:
I20260709 13:13:38.916038   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.915910   924 rpcz_store.cc:276] 0709 13:13:36.431281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.895440 (+2464159us) service_pool.cc:224] Handling call
0709 13:13:38.914246 (+ 18806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.915169 (+   923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.915400 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.807790  3447 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.841470  3338 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.821416  3429 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.869386   851 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:38.846046  3359 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.803244   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.846001  3183 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.893553  3263 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:38.798823  3386 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.875530   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.828167  3142 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.885655   922 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.836943  3298 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:13:38.870230  3258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.019s	sys 0.038s
I20260709 13:13:38.820906   847 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.828296  3247 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.889325   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.891774   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:38.904639  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55924us
I20260709 13:13:38.906091  3248 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:38.886546  3187 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.910799   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.955914  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:38.956585   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.957719   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:38.896438  3404 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.518s	user 0.004s	sys 0.005s
I20260709 13:13:38.909307  3382 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.954272   850 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.914489  3423 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.922122  3338 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.918090  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
W20260709 13:13:38.920545  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 200589us
I20260709 13:13:38.894567  3250 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:38.799117  3377 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:38.952683   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 77231.2us
I20260709 13:13:38.907649   930 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:38.889974  3272 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.953639   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.978796   847 raft_consensus.cc:2393] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 3: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:38.888057  3305 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.934126  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 306) took 1632 ms. Trace:
I20260709 13:13:38.905607  3320 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:38.991726  3320 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:38.993170  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:38.964316  3142 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:38.970381  3423 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.895159  3309 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.948683   749 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 3 to 4, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:38.961732   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.975929  3250 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:38.964339  3386 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:38.922873  3401 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:38.965488   926 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:38.966439  3381 consensus_meta.cc:287] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.003s
I20260709 13:13:38.919828  3111 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:38.832492  3361 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.969643   922 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
W20260709 13:13:38.970414   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 150547us
I20260709 13:13:39.063696   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.065160   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:38.972368   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:38.971727  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:39.066738  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95135.3us
I20260709 13:13:38.952814  3263 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.977128  1001 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:38.977414   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1535) took 2357 ms. Trace:
I20260709 13:13:39.070950   930 rpcz_store.cc:276] 0709 13:13:36.619789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.898009 (+2278220us) service_pool.cc:224] Handling call
0709 13:13:38.907228 (+  9219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.977147 (+ 69919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.977352 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:38.912910   896 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:38.959057  2959 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.975124   999 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:38.964274  3428 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.980042   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.968307  3382 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:38.988348  3096 consensus_meta.cc:287] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.002s
I20260709 13:13:38.981619  1597 rpcz_store.cc:276] 0709 13:13:37.301607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.308337 (+  6730us) service_pool.cc:224] Handling call
0709 13:13:38.034683 (+726346us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:38.933849 (+899166us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400331b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:38.934043 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664961024}
I20260709 13:13:38.911717  3307 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:38.919644  3356 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:38.973182  3447 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.033762   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:38.960729  3440 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.058214  3111 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:39.079368  2959 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:39.113529  3111 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:39.113739  2959 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.069567  3142 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:39.112114  3440 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.117125  3440 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.112262  3447 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.121868  2959 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.086905  3183 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:39.123963  3183 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.124320  3183 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.125538  3331 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:39.126837  2959 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.127413  3344 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:39.127904  3344 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.129195  3356 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:38.968622  3404 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.131124  3404 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:39.131757   849 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
W20260709 13:13:39.132545   925 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1538) took 2276 ms (client timeout 1858 ms). Trace:
W20260709 13:13:39.132900   925 rpcz_store.cc:269] 0709 13:13:36.856486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.649346 (+792860us) service_pool.cc:224] Handling call
0709 13:13:39.132282 (+1482936us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54003962d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:39.132470 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440210816}
I20260709 13:13:39.133195   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.069796   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.069051   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.134156   925 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:39.134810   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1550) took 1916 ms. Trace:
W20260709 13:13:39.134861   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 65897.6us
W20260709 13:13:39.079852   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 167009us
I20260709 13:13:39.080258   930 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:39.136085   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1549) took 1918 ms. Trace:
I20260709 13:13:39.052057  3401 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.136509   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.136739  3401 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.137679   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:39.139042   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.086391   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.139854  3331 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:39.112656   850 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:39.140300  3331 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:39.140786  3331 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:39.141090  3439 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.859s	user 0.004s	sys 0.003s
I20260709 13:13:39.141043   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.140022   847 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:39.141873  3363 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:39.142387  3363 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:39.142462   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.140066  3448 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:39.143679  3448 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:39.140184  3404 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:39.144845  3404 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:39.143293  3363 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.145711  3404 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.123205  3248 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.140022   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.148643  3111 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.149333   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.150038  3142 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:39.150211  3248 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:39.135154   925 rpcz_store.cc:276] 0709 13:13:37.218174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.133658 (+1915484us) service_pool.cc:224] Handling call
0709 13:13:39.133956 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.134571 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.134764 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.136387   930 rpcz_store.cc:276] 0709 13:13:37.217905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.071880 (+1853975us) service_pool.cc:224] Handling call
0709 13:13:39.079817 (+  7937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.135850 (+ 56033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.136035 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.069720  3361 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:39.153267   923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1552) took 1699 ms (client timeout 1881 ms). Trace:
I20260709 13:13:39.153842   848 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:39.152041   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1554) took 1697 ms. Trace:
I20260709 13:13:39.154666   925 rpcz_store.cc:276] 0709 13:13:37.454500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.151258 (+1696758us) service_pool.cc:224] Handling call
0709 13:13:39.151539 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.151720 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.151744 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.151788 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:39.151830 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.151887 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.151992 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:39.092412   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.140941   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.157264   850 raft_consensus.cc:2410] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:39.155184   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.158301   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.159266   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.144415  3448 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.152369   930 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:39.150671  3111 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:39.153640   923 rpcz_store.cc:269] 0709 13:13:37.454106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.298585 (+844479us) service_pool.cc:224] Handling call
0709 13:13:39.153117 (+854532us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b54001699d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:39.153217 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35816192}
I20260709 13:13:39.072502  3144 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:39.163972   925 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:39.184718   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1649) took 1713 ms. Trace:
I20260709 13:13:39.185185   925 rpcz_store.cc:276] 0709 13:13:37.470966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.155665 (+1684699us) service_pool.cc:224] Handling call
0709 13:13:39.155929 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.184367 (+ 28438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.184644 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.186604   925 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:39.169720   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.187354   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1654) took 1689 ms. Trace:
I20260709 13:13:39.087338  3215 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.196023   925 rpcz_store.cc:276] 0709 13:13:37.498094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.186038 (+1687944us) service_pool.cc:224] Handling call
0709 13:13:39.186386 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.187101 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.187306 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.198249   852 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:39.198084  3361 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.199841  3430 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:39.200240  3430 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:39.200696  3361 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:39.201022  3430 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.202335  3430 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.167354   851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:39.204741   851 raft_consensus.cc:606] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.205075  3377 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.206838  3305 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.168056  3142 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:39.208384  3448 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.213594  1566 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:39.205091   851 raft_consensus.cc:614] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:39.217139   848 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:39.134181   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.218686   849 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
W20260709 13:13:39.186538  3449 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:13:39.060137  3381 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.084306  3457 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:39.109246  3096 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.032756  3309 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:39.183393   850 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:39.165477   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1645) took 1695 ms. Trace:
I20260709 13:13:39.187580  3248 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:39.187825   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:39.253569   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 65839.7us
I20260709 13:13:39.189275  3344 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.194155   923 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:39.256290   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1650) took 1785 ms. Trace:
W20260709 13:13:39.177335   967 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.858s (SENT)
I20260709 13:13:39.185002  3298 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:39.223989   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.225329   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1656) took 1705 ms. Trace:
I20260709 13:13:39.257297   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.257788   925 rpcz_store.cc:276] 0709 13:13:37.520309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.196857 (+1676548us) service_pool.cc:224] Handling call
0709 13:13:39.197128 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.224830 (+ 27702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.224900 (+    70us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.224944 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:39.224998 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.225096 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.225247 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27375,"raft.queue_time_us":35470,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11358}
I20260709 13:13:39.259778   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.086268  3335 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:39.261152   925 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:39.262904   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.265836   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:39.266407   850 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:39.231853  3305 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:39.268694   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.249656   930 rpcz_store.cc:276] 0709 13:13:37.469740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.151734 (+1681994us) service_pool.cc:224] Handling call
0709 13:13:39.152060 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.165159 (+ 13099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.165410 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.256757   923 rpcz_store.cc:276] 0709 13:13:37.471252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.193525 (+1722273us) service_pool.cc:224] Handling call
0709 13:13:39.193844 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.255975 (+ 62131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.256223 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:39.247336   968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.019s	sys 0.039s
W20260709 13:13:39.271816   968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.019s	sys 0.039s
I20260709 13:13:39.265400   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.237073  3111 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.275070   928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 269) took 1953 ms (client timeout 1955 ms). Trace:
W20260709 13:13:39.273437  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 489214us
I20260709 13:13:39.273614   995 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:13:39.279490   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 102210us
W20260709 13:13:39.280203   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:39.280757   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:39.281025   967 connection.cc:256] RPC call timeout handler was delayed by -1.3272024099999271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:13:39.281934  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:39.275570   928 rpcz_store.cc:269] 0709 13:13:37.321604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.965834 (+644230us) service_pool.cc:224] Handling call
0709 13:13:39.274874 (+1309040us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54002314d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:39.275005 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334471552}
I20260709 13:13:39.283521   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.284230   928 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:39.257002  3322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.607s	user 0.025s	sys 0.021s
I20260709 13:13:39.285681  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.287124  3361 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.286635  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:39.285758  3434 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:39.287734  3361 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.289361   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.289788  1597 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:39.290488  3341 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:39.290730   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.290920   969 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:39.287693  3322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.638s	user 0.025s	sys 0.021s
I20260709 13:13:39.287075  3356 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:13:39.291991  3364 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:39.292439  3364 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:39.292479  3356 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:39.292918  3364 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.293030  3356 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:39.294118  3356 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.296414   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.298242  3096 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:39.299324  3401 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.299749  3247 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:39.301450  3401 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.302839   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.303968   925 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:39.303670  3331 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.249655  3457 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.306362  3457 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:39.306474  3331 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.287051  3458 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:39.254565  3309 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.308641   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.308822  3305 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:39.309525  3111 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.310273  3305 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.252964  3248 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.311889  3111 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.260704  3298 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:39.267997  3451 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:13:39.312649  3298 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:39.314046  3465 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:13:39.311998  3248 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.320005  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:39.325287   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.326527  3429 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.269045  3431 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:39.328063   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.340611   997 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:39.341777   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.328519  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.342854  1001 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:39.344005  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.344591  1001 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:39.345635  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.346302  1001 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:39.330005  1004 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.332479  1564 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.347502   997 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.348023  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:39.353868   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.354579  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.337986  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.355888  1000 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:39.337888  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:39.356802  1004 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
W20260709 13:13:39.338156   926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 267) took 2018 ms (client timeout 1805 ms). Trace:
W20260709 13:13:39.357761   926 rpcz_store.cc:269] 0709 13:13:37.320509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.916345 (+595836us) service_pool.cc:224] Handling call
0709 13:13:39.337928 (+1421583us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400082854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:39.338074 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217559936}
I20260709 13:13:39.338763  1003 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:39.338181   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.269095  3335 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.336699   850 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:39.360491   849 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:39.361093   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.365085   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:39.370028   850 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.362296   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:39.316118  2959 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:39.289391  3361 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.383649  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.397146  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:39.397468  1003 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.390856   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 270) took 1582 ms. Trace:
I20260709 13:13:39.269187  3377 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:13:39.401134  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:39.404294  1595 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:39.337834  1594 raft_consensus.cc:2368] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4.
I20260709 13:13:39.340070  3356 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.402233  3431 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:39.398753  3341 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:39.416296  3431 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.419356   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.420111   998 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:39.421133   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.421890   998 raft_consensus.cc:2368] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:13:39.422690   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:39.423025   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.399190  2959 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.423936  3097 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.424175  2959 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.424510  3097 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.389273  3335 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.432277   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.447240   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.448051   998 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:39.449458   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 476527us
W20260709 13:13:39.450850   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.398303   926 rpcz_store.cc:276] 0709 13:13:37.808162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.390033 (+1581871us) service_pool.cc:224] Handling call
0709 13:13:39.390325 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.390523 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.390548 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.390591 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:39.390622 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.390678 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.390802 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.469470   849 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:39.474251   850 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:39.391773  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.394733   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.394949  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:39.395510  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 424169us
I20260709 13:13:39.485090  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:39.488991   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 94331us
I20260709 13:13:39.489883   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.491039   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.388669   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:39.492609   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 104066us
I20260709 13:13:39.493247   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.493719  1001 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:39.494104   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:39.495278   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.496922   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.500295  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.417173  3341 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:39.503563  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.505570  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.506146  1004 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:39.499965  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:39.506934  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.507993  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.508630  1004 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.462882  3097 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:39.509618  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.509891   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.472375  3457 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:39.476390   928 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:39.518734   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.519788   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.520398   928 raft_consensus.cc:2376] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6. Re-sending same reply.
I20260709 13:13:39.521514  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.521811   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.522133  1000 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:39.525303  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.529744  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.494992  3401 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.504801  3386 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.504663  3347 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:39.540136  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.515516  3111 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:39.540812  1000 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:39.520038  3456 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:39.545490   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.545398  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.495321  3364 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.509671  3258 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:13:39.534476  3457 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:39.540905  3363 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.545025  3111 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:39.558542  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.539886  3386 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.561810   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.542438  3347 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.569485   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.557586  3381 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:39.391238   929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.015s	sys 0.021s
W20260709 13:13:39.582952   929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.015s	sys 0.021s
I20260709 13:13:39.584024   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 265) took 3214 ms. Trace:
I20260709 13:13:39.552956  1003 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:39.551060  3456 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:39.551891  3331 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:39.580749  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:39.578272   998 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:39.587071  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.597594  1003 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:39.574568  3403 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.004s	sys 0.017s
I20260709 13:13:39.593765  3382 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.593773  3183 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.596634  1000 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:13:39.608855  3263 consensus_meta.cc:287] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.002s
I20260709 13:13:39.612756  1592 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:39.615840  3359 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 13:13:39.617066  3344 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:39.618199  3460 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:39.618762  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:13:39.576833  3111 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:39.577791  3386 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.582132  3347 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.588792  3428 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.630178   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:39.631476  1001 raft_consensus.cc:606] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.644294   853 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:39.599061   997 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:39.653672   998 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.595395  3456 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:39.657615  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 309628us
W20260709 13:13:39.600382  3403 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.004s	sys 0.017s
W20260709 13:13:39.632838  3250 consensus_meta.cc:287] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.002s
W20260709 13:13:39.668777  3459 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:13:39.599149  3447 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.674929  1599 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:39.658999  3250 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.676280  1564 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.677177  3250 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.677446  3399 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:39.586103  3341 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.690128  1001 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:39.649086  3307 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:39.691936  1001 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:39.627460  3263 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.635526  3359 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.700906 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:39.687880  3250 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.695187  3399 raft_consensus.cc:697] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:39.742115   847 raft_consensus.cc:2393] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:39.743084  1003 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:39.746379  1004 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:39.751075  1003 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:39.753629   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 329273us
I20260709 13:13:39.662051  3457 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.763540  1004 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:39.742029  3183 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:39.774243   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 263260us
I20260709 13:13:39.757366  3427 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:13:39.753032  3111 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.808715  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:39.783742   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 232313us
W20260709 13:13:39.775383   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 281312us
W20260709 13:13:39.911886  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 103287us
W20260709 13:13:39.912042   967 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 2.094s (SENT)
W20260709 13:13:39.912500   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 415608us
I20260709 13:13:39.912798   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:39.858592   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.775807   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.798427   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:39.807991   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:39.915814   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:39.849119   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1548) took 2631 ms. Trace:
I20260709 13:13:39.915418   851 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.917497   922 rpcz_store.cc:276] 0709 13:13:37.217644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.061060 (+1843416us) service_pool.cc:224] Handling call
0709 13:13:39.061527 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.061943 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.061983 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.081556 (+ 19573us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.546485 (+464929us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:39.546555 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.546651 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.848757 (+302106us) raft_consensus.cc:1685] finished
0709 13:13:39.848794 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.849023 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":16890,"thread_start_us":360388,"threads_started":1,"wal-append.queue_time_us":600455}
I20260709 13:13:39.913354   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:39.917042   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 141332us
I20260709 13:13:39.920857   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.926338   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:39.854758  3338 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.876s	user 0.008s	sys 0.004s
I20260709 13:13:39.928041   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.947640   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.948501   922 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:39.929179   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:39.952275   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:39.766608  3280 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.949596   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:39.796193  3467 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:13:39.867810  3344 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:39.791989  3399 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.961016   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:39.962249   821 leader_election.cc:400] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:13:39.963083   821 leader_election.cc:403] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:39.965397   922 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:39.793608  3364 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:39.925885  3440 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.007s
I20260709 13:13:39.958652   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:39.969677   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.837240  3382 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.970921   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.980361   922 raft_consensus.cc:2376] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4. Re-sending same reply.
I20260709 13:13:39.926580  3404 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:39.971686   851 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:39.754343  3250 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:39.977396   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65203.7us
I20260709 13:13:39.975906   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.986207   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.986876   925 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6.
I20260709 13:13:39.988075   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:39.988806   925 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.982806   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:39.929337  3228 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:39.991631   922 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:39.968144  3434 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:39.799293  3248 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.959975  3458 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.798486  2959 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.802727  3298 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:39.984755  3404 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:39.974350  3263 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.974835  3322 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.976094  3300 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:39.981323  3479 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.976228  3144 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:39.978189  3483 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.978974  3477 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:40.000828  3305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.035s	sys 0.039s
I20260709 13:13:39.975876  3403 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:40.000579  3364 raft_consensus.cc:2764] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:39.973265  3344 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.978538  3142 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:13:39.978057  3280 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:39.971431  3338 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.010273  3404 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.978755  3440 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.013175  3322 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:40.026755  3305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.037s	sys 0.040s
I20260709 13:13:40.019624  3429 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:40.040561  3356 consensus_meta.cc:287] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.001s
W20260709 13:13:40.040088  3452 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.068s	user 0.003s	sys 0.003s
W20260709 13:13:40.046278  3309 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.002s
I20260709 13:13:40.046686  3440 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.983652  3482 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:13:40.039824  3322 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.024634  3481 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:13:40.046531  3347 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.062716  3347 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.026290  3479 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.065701  3344 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:40.067793  3423 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.020s	user 0.010s	sys 0.001s
W20260709 13:13:40.069211  3361 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.001s
I20260709 13:13:40.202925  3361 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.206364  3361 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:39.969354   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.208134  3344 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:40.209255   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 239886us
I20260709 13:13:40.210767   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:40.211829  3344 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:40.212150   930 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:40.115187   849 raft_consensus.cc:2368] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6.
I20260709 13:13:40.116822   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1718) took 1716 ms. Trace:
I20260709 13:13:40.116778   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.005735   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:40.212954   854 rpcz_store.cc:276] 0709 13:13:38.400472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.402278 (+  1806us) service_pool.cc:224] Handling call
0709 13:13:38.431414 (+ 29136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.431851 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.431884 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.458843 (+ 26959us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.867159 (+408316us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:38.867200 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.867263 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.925993 (+1058730us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:40.115932 (+189939us) raft_consensus.cc:1685] finished
0709 13:13:40.115998 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.116555 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170517,"raft.queue_time_us":476,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":24791,"spinlock_wait_cycles":15114496,"thread_start_us":274319,"threads_started":1,"wal-append.queue_time_us":478899}
I20260709 13:13:40.216260   922 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:40.217777   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.217908   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:40.218725   854 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.219069   922 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.218849   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.131762   927 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.132953   928 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:40.002635   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 88886.2us
I20260709 13:13:40.222682   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.222815   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:39.584573   929 rpcz_store.cc:276] 0709 13:13:36.369228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.665123 (+2295895us) service_pool.cc:224] Handling call
0709 13:13:38.665374 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.665560 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:38.878029 (+212469us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.583526 (+705497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.583596 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:39.583713 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.583801 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.583953 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299311,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":2135,"thread_start_us":865119,"threads_started":2,"wal-append.queue_time_us":652001}
W20260709 13:13:40.133165   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 146902us
I20260709 13:13:40.228550   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:40.139779   926 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.232631  3363 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.233619  3363 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:40.234006  3363 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.234337  3363 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:40.030375  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 102863us
I20260709 13:13:40.236928   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.237653   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1503) took 1940 ms. Trace:
I20260709 13:13:40.237931   850 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.165405  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:40.238123   995 rpcz_store.cc:276] 0709 13:13:38.297028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.299478 (+  2450us) service_pool.cc:224] Handling call
0709 13:13:38.299796 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.276949 (+977153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.237029 (+960080us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b54000ea954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:40.237436 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233089920}
I20260709 13:13:40.238822  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:40.240139  3399 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:40.242491   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.026980  3097 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:40.177157  3248 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.177639  3111 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.178397  2959 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.180269  3280 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.180337  3250 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.182253  3298 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.069271  3404 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.073822  3423 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:40.075836  3430 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.873s	user 0.012s	sys 0.003s
I20260709 13:13:40.040089  3456 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.076119  3479 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.183292  3309 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.189321  3429 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:13:40.193271  3335 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.001s
W20260709 13:13:40.196326  3386 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.002s
I20260709 13:13:40.212344  3344 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:40.214604   851 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:40.130246   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1547) took 2912 ms. Trace:
I20260709 13:13:40.243510  3399 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:40.300309  3399 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:40.285075   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:40.302425  3399 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:40.303088   927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 273) took 1591 ms (client timeout 2061 ms). Trace:
I20260709 13:13:40.303756  3399 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:40.303718   927 rpcz_store.cc:269] 0709 13:13:38.711443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.848354 (+136911us) service_pool.cc:224] Handling call
0709 13:13:40.302807 (+1454453us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001b0654. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.303005 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821216640}
I20260709 13:13:40.304903  3272 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.213155  3338 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.231148   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:40.305830   927 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.306178   848 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:40.306604   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1665) took 1999 ms. Trace:
I20260709 13:13:40.304023  3456 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.307689  3429 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:40.297652   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.248699   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.309158  3344 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:40.309860   923 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:40.310046   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.310992   852 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:40.269093  3307 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:40.292526  3494 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:40.245451  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:40.312886  3494 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.312949   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1734) took 1046 ms. Trace:
I20260709 13:13:40.281513  1597 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:40.315665   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.314172   924 rpcz_store.cc:276] 0709 13:13:37.217138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.923855 (+1706717us) service_pool.cc:224] Handling call
0709 13:13:38.961814 (+ 37959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.962251 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.962294 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.977485 (+ 15191us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.520770 (+543285us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:39.906528 (+385758us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.975527 (+ 68999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.975653 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.129511 (+153858us) raft_consensus.cc:1685] finished
0709 13:13:40.129577 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.129928 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88855,"prepare.queue_time_us":513834,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":71104,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":5815,"thread_start_us":841915,"threads_started":2,"wal-append.queue_time_us":587613}
I20260709 13:13:40.287887  3187 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.253926  1596 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:40.314405  3097 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.312386  3307 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.319242  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 479) took 1034 ms. Trace:
I20260709 13:13:40.307116   927 rpcz_store.cc:276] 0709 13:13:38.306786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.305161 (+1998375us) service_pool.cc:224] Handling call
0709 13:13:40.305555 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.306260 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.306506 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.313359   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.274174  3428 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:40.308442  3429 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:40.307119   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 76228.6us
W20260709 13:13:40.214936   846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1539) took 1351 ms (client timeout 1709 ms). Trace:
I20260709 13:13:40.298790   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1550) took 1377 ms. Trace:
W20260709 13:13:40.328653   846 rpcz_store.cc:269] 0709 13:13:38.856345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.874303 (+ 17958us) service_pool.cc:224] Handling call
0709 13:13:40.207058 (+1332755us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540041f0d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.207315 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2869246848}
W20260709 13:13:40.332666   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 106306us
I20260709 13:13:40.314239   848 rpcz_store.cc:276] 0709 13:13:39.266407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.421735 (+155328us) service_pool.cc:224] Handling call
0709 13:13:40.305515 (+883780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.306772 (+  1257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.312750 (+  5978us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54004092d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.312875 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031372032}
W20260709 13:13:40.244297  1002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33108 (request call id 276) took 1796 ms (client timeout 1605 ms). Trace:
W20260709 13:13:40.283149  3462 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.712s	user 0.002s	sys 0.000s
I20260709 13:13:40.343384  3494 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.334214  2959 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.345384  2959 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.352298   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1519) took 1888 ms. Trace:
I20260709 13:13:40.352932   855 rpcz_store.cc:276] 0709 13:13:38.463925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.476736 (+ 12811us) service_pool.cc:224] Handling call
0709 13:13:38.477052 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.484972 (+  7920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.485010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.488268 (+  3258us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.945855 (+457587us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b180026bae0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:13:39.271308 (+325453us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:39.292608 (+ 21300us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.293045 (+   437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.293111 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.281931 (+988820us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:40.351723 (+ 69792us) raft_consensus.cc:1685] finished
0709 13:13:40.351756 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.352040 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41310,"prepare.queue_time_us":20891,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":189533,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":24940,"spinlock_wait_cycles":12219648,"thread_start_us":325268,"threads_started":1,"wal-append.queue_time_us":876730}
I20260709 13:13:40.360911  3344 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.363390  1601 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:40.332098   853 rpcz_store.cc:276] 0709 13:13:38.921602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.069469 (+147867us) service_pool.cc:224] Handling call
0709 13:13:40.298289 (+1228820us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54004092d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.298586 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171872512}
I20260709 13:13:40.363849  3335 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.365002   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:40.366551  3335 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.367426  3386 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.276870  3447 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.245430   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.312129  3404 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.366466   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:40.274909  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.369547  3344 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.387476  3479 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
W20260709 13:13:40.334896  1002 rpcz_store.cc:269] 0709 13:13:38.447402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.447536 (+   134us) service_pool.cc:224] Handling call
0709 13:13:40.243726 (+1796190us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54000e9f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.244048 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2050067328}
I20260709 13:13:40.554555  3386 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.557492  3280 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:40.558346   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1681) took 1301 ms. Trace:
I20260709 13:13:40.558769   930 rpcz_store.cc:276] 0709 13:13:39.256743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.296146 (+ 39403us) service_pool.cc:224] Handling call
0709 13:13:40.558101 (+1261955us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.558281 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103533824}
I20260709 13:13:40.556980   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1735) took 1290 ms. Trace:
I20260709 13:13:40.559751  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:40.560943  1597 raft_consensus.cc:2368] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5.
I20260709 13:13:40.558777  3280 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.560319   855 rpcz_store.cc:276] 0709 13:13:39.266687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.555802 (+1289115us) service_pool.cc:224] Handling call
0709 13:13:40.556342 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.556641 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.556663 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.556699 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:40.556724 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.556783 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.556904 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.496690   854 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:40.376734   921 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:13:40.552416   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 192277us
I20260709 13:13:40.565147   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:40.246177  1594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 475) took 1895 ms (client timeout 1791 ms). Trace:
W20260709 13:13:40.496090   925 consensus_meta.cc:287] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.012s
I20260709 13:13:40.383087   851 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:40.243912  3482 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.391754   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.492624   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1666) took 2181 ms. Trace:
W20260709 13:13:40.491518   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.476163   924 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:40.281384  3359 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.345327  3494 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:40.574083   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 331796us
I20260709 13:13:40.546921  3272 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:40.335218  3428 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.577775  3427 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:40.579077   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 71565.2us
I20260709 13:13:40.576598   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1667) took 2263 ms. Trace:
I20260709 13:13:40.581826   924 rpcz_store.cc:276] 0709 13:13:38.313093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.317166 (+2004073us) service_pool.cc:224] Handling call
0709 13:13:40.317770 (+   604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.576335 (+258565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.576550 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:40.583012  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 308475us
I20260709 13:13:40.319730  1596 rpcz_store.cc:276] 0709 13:13:39.284972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.327159 (+ 42187us) service_pool.cc:224] Handling call
0709 13:13:40.252725 (+925566us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ae9c 00007f0cffa1aa5b 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:40.253236 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.318908 (+ 65672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.319149 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35644672}
I20260709 13:13:40.584668  3493 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:40.589520   968 leader_election.cc:400] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5.
I20260709 13:13:40.589833   968 leader_election.cc:403] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:40.589246   852 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:40.572000  3456 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.574103   929 rpcz_store.cc:276] 0709 13:13:38.311481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.316297 (+2004816us) service_pool.cc:224] Handling call
0709 13:13:40.316777 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.317179 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:40.317627 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.492019 (+174392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.492088 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:40.492148 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.492243 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.492407 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81612,"prepare.queue_time_us":446,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":82638,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":122,"thread_start_us":92147,"threads_started":1}
I20260709 13:13:40.378171  3423 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.582682   925 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:40.596073   855 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.592546   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1664) took 2270 ms. Trace:
I20260709 13:13:40.538262  3307 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.579636  3338 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:40.489362  3364 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:40.497431  3363 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.602591   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 256) took 1098 ms. Trace:
I20260709 13:13:40.604432  3309 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.609443   855 rpcz_store.cc:276] 0709 13:13:39.499416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.567403 (+1067987us) service_pool.cc:224] Handling call
0709 13:13:40.595842 (+ 28439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.598013 (+  2171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.598205 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:40.609809   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 389094us
I20260709 13:13:40.613588   922 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:40.614228   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1669) took 2298 ms. Trace:
I20260709 13:13:40.614545   922 rpcz_store.cc:276] 0709 13:13:38.316232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.319915 (+2003683us) service_pool.cc:224] Handling call
0709 13:13:40.581713 (+261798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.614002 (+ 32289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.614178 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.615991  3331 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:40.616055   922 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:40.616941   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49151.9us
I20260709 13:13:40.618783   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.602279  3493 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:40.627548   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1558) took 1233 ms. Trace:
I20260709 13:13:40.637001   923 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
W20260709 13:13:40.568547  1594 rpcz_store.cc:269] 0709 13:13:38.350968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.355136 (+  4168us) service_pool.cc:224] Handling call
0709 13:13:40.245563 (+1890427us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400122fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.245889 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025847040}
I20260709 13:13:40.636778   922 rpcz_store.cc:276] 0709 13:13:39.391312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.615204 (+1223892us) service_pool.cc:224] Handling call
0709 13:13:40.615557 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.616692 (+  1135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.624650 (+  7958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.634418   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:40.641744   850 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:40.623744  3331 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.638540   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.642426   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1567) took 1244 ms. Trace:
I20260709 13:13:40.620267  3338 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.627342  3427 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.635175  2959 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.635174  3404 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.596961   928 rpcz_store.cc:276] 0709 13:13:38.306512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.300177 (+1993665us) service_pool.cc:224] Handling call
0709 13:13:40.300732 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.301193 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:40.301685 (+   492us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.569957 (+268272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.570013 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:40.574155 (+  4142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.574225 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.577164 (+  2939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17213,"prepare.queue_time_us":6325,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":64,"raft.queue_time_us":92983,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":166,"thread_start_us":109489,"threads_started":1,"wal-append.queue_time_us":276016}
I20260709 13:13:40.645352 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:40.646270   923 rpcz_store.cc:276] 0709 13:13:39.393674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.248282 (+854608us) service_pool.cc:224] Handling call
0709 13:13:40.637786 (+389504us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.651242   848 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:40.633061  3498 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:40.624212  3456 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:40.656841   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 290545us
I20260709 13:13:40.658789   848 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:40.657884   749 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 3 to 5, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 5 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.656154   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1671) took 2333 ms. Trace:
I20260709 13:13:40.636798  3097 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.655596   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:40.666831   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.670703   925 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.668041   930 rpcz_store.cc:276] 0709 13:13:38.319209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.567350 (+2248141us) service_pool.cc:224] Handling call
0709 13:13:40.600045 (+ 32695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.648548 (+ 48503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.648576 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.652178 (+  3602us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:40.652272 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.652349 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.652462 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5201,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7986}
I20260709 13:13:40.647553  3331 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:40.680029   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 68887.1us
I20260709 13:13:40.684680   929 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:13:40.678611   926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1568) took 1284 ms (client timeout 1548 ms). Trace:
I20260709 13:13:40.687850   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:40.695745  3429 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.699028   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1535) took 1825 ms. Trace:
I20260709 13:13:40.699544   847 rpcz_store.cc:276] 0709 13:13:38.828521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.142222 (+313701us) service_pool.cc:224] Handling call
0709 13:13:40.626990 (+1484768us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400409cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.653968 (+ 26978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376292608}
I20260709 13:13:40.700917   850 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:40.702104   850 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:40.703390   850 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:13:40.709542  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.712867   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 255) took 1397 ms. Trace:
I20260709 13:13:40.713397   853 rpcz_store.cc:276] 0709 13:13:39.315702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.692796 (+1377094us) service_pool.cc:224] Handling call
0709 13:13:40.712170 (+ 19374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.712503 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.712537 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.712586 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:40.712626 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.712691 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.712802 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.673635  3280 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:40.692891   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.687765   926 rpcz_store.cc:269] 0709 13:13:39.393914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.461235 (+ 67321us) service_pool.cc:224] Handling call
0709 13:13:40.674553 (+1213318us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003282d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.678550 (+  3997us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435130112}
I20260709 13:13:40.672767  3482 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:40.721627   854 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.727968  3214 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.728561  3214 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.729756   848 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.730823  3214 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.739748  3493 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:40.740886   967 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.578s (SENT)
I20260709 13:13:40.741379   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.743571  3455 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:40.678610  3427 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.747308  3427 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.675159  3498 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:40.756126  3498 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.757975  3363 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:40.763329   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:40.763808  3455 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:40.764854  3455 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:40.765177  3455 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:40.766240  3455 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.769516  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.770335  3322 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.771675  3477 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:40.771780  3399 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:40.772594  3399 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:40.772799  3477 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:40.773744  3399 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.774657  3477 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.754297  3338 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.682614  3423 raft_consensus.cc:2764] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:40.775068  3477 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:13:40.783232   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 62224us
I20260709 13:13:40.797312   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.803902   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:40.799527   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:40.802537   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1674) took 2462 ms. Trace:
W20260709 13:13:40.817646   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.672586  3364 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.814831   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.823958  3455 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:40.827818   967 leader_election.cc:336] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.583s (SENT)
I20260709 13:13:40.828524   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:40.830001   967 leader_election.cc:336] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.592s (SENT)
I20260709 13:13:40.830454   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:40.831801  3096 consensus_queue.cc:1175] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64f2cb5b0e174002811ef197f9eec829" responder_term: 6 status { last_received { term: 6 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 5 } }
W20260709 13:13:40.835341   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 55064.1us
I20260709 13:13:40.836120  3307 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.836510  3456 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.837239   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.837435   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:40.838248  3401 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:40.838990   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 161479us
I20260709 13:13:40.839789   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:40.841627   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 279) took 1093 ms. Trace:
I20260709 13:13:40.842020   922 rpcz_store.cc:276] 0709 13:13:39.748428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.644942 (+896514us) service_pool.cc:224] Handling call
0709 13:13:40.684028 (+ 39086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.821968 (+137940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.822011 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.841363 (+ 19352us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:40.841403 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.841464 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.841572 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:40.846953   926 raft_consensus.cc:2393] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.839340  3322 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:40.851922   851 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:40.857631   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:40.858292   925 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:40.859699   849 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.858466   850 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:40.863327   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.864355  3496 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:40.865235  3496 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:40.867758  3341 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:40.868561  3272 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.869074  3272 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.872509  3341 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.872956  3341 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:40.873831  3341 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.875128  3272 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:40.878891   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:40.879734   846 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:40.880177   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1557) took 1402 ms. Trace:
I20260709 13:13:40.880478   846 rpcz_store.cc:276] 0709 13:13:39.477883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.779875 (+1301992us) service_pool.cc:224] Handling call
0709 13:13:40.880091 (+100216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:40.881302  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 203610us
I20260709 13:13:40.863341  3390 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:40.882548  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:40.882959  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:40.883850  1592 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:40.884104   847 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.884447   852 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:40.891811  3359 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:40.903532   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 306759us
I20260709 13:13:40.903481   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1670) took 2578 ms. Trace:
I20260709 13:13:40.908484  3479 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:40.909142  3477 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.909181  3479 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.910282  3477 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:40.911836  3479 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.912258  3428 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.904289  3097 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:40.914381  3386 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:40.914552  3097 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.915131  3097 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.915045  3386 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.917253  3386 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:40.918540   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78928.4us
I20260709 13:13:40.920854  3364 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.921882   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.915222   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:40.905042  3401 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:40.943349   930 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:40.905892  3390 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.906416   927 rpcz_store.cc:276] 0709 13:13:38.318082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.336343 (+2018261us) service_pool.cc:224] Handling call
0709 13:13:40.745643 (+409300us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b5800010598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:40.745859 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.746144 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:40.746490 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.896379 (+149889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.896433 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:40.896478 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.896541 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.896642 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43782,"prepare.queue_time_us":194,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":53489,"spinlock_wait_cycles":4315392,"thread_start_us":101722,"threads_started":1,"wal-append.queue_time_us":158651}
I20260709 13:13:40.955571  3214 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:40.816417   924 rpcz_store.cc:276] 0709 13:13:38.339832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.587429 (+2247597us) service_pool.cc:224] Handling call
0709 13:13:40.587897 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.588217 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:40.601272 (+ 13055us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.784945 (+183673us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.785104 (+   159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.785150 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:40.802275 (+ 17125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.802347 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.802468 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24266,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":29480,"spinlock_wait_cycles":11888000,"thread_start_us":130771,"threads_started":1}
I20260709 13:13:40.956575  3359 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:40.956734  3214 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.957402  3214 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:40.958371  3214 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.959816  3214 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:40.959914  1596 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:40.961033  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.965945   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:40.968638  1000 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.970038  3401 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.971861  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.973384  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.974226  3493 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:40.974607  3493 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.974941  3493 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:40.975011   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1647) took 3504 ms. Trace:
I20260709 13:13:40.975569   921 rpcz_store.cc:276] 0709 13:13:37.470374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.153072 (+1682698us) service_pool.cc:224] Handling call
0709 13:13:39.153396 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.574095 (+1420699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.974678 (+400583us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003cbc54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:40.974945 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812052736}
I20260709 13:13:40.979311   928 raft_consensus.cc:2410] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:40.979794  3280 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:40.979743   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:40.980456  3280 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:40.980973   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:40.981175  3390 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.981882  3482 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:40.982313  3482 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:40.971314  3401 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.982971  3482 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.984886  3428 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:40.943553  3111 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:40.986164  3111 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:40.986515  3111 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:40.987319  3111 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:40.989034   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1682) took 1692 ms. Trace:
I20260709 13:13:40.982271   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.989418   929 rpcz_store.cc:276] 0709 13:13:39.296340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.606534 (+1310194us) service_pool.cc:224] Handling call
0709 13:13:40.679664 (+ 73130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.687147 (+  7483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.988836 (+301689us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b54003282d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:40.988981 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99079936}
I20260709 13:13:40.981477  3280 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:40.992937   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:40.993623   928 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.994369   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:40.995012   995 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:40.995371  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:40.996062   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.997447   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:40.999907  3483 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:41.000376  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47756.1us
I20260709 13:13:41.000808  3483 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.001284   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.001587  3483 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:40.993038  3335 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:13:41.007997   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 169768us
I20260709 13:13:41.005050   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:41.005788  2959 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:13:41.006389  3512 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:13:40.998633  3097 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.004825  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.009197   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1553) took 1526 ms. Trace:
I20260709 13:13:41.000844   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.000209   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:40.988752  3111 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:41.015944   818 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:41.013684   849 rpcz_store.cc:276] 0709 13:13:39.476954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.703054 (+1226100us) service_pool.cc:224] Handling call
0709 13:13:40.859450 (+156396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.860149 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.003285 (+143136us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001ccad0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:41.003413 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342849792}
I20260709 13:13:41.021297   999 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.022164   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.022931   923 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.008407  3335 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.010725  2959 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.020952   995 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:41.007655  3493 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.029206  1002 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:41.017220  3429 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:41.029316  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:41.031893   967 leader_election.cc:336] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.849s (SENT)
I20260709 13:13:41.023717  3214 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:41.030309  2959 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.044937   969 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:41.050016   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.052767  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.072327   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:41.073937  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50317.4us
I20260709 13:13:41.074167  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.081664   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.087781  1601 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:41.088869  3341 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.089313   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:41.096158   999 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
W20260709 13:13:41.103931   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 76360.4us
W20260709 13:13:41.105865   967 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.872s (SENT)
I20260709 13:13:41.106431   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:41.108616  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:41.109212   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.109499  1001 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:41.109910   995 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.112227   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:41.117905  1592 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:13:41.117359  3335 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.118971  3448 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:13:41.120083  3214 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:41.120446  3214 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.121416  3214 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.122895   995 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:41.123025  3214 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:41.128719  3493 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:41.129540   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47624.5us
W20260709 13:13:41.135226  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.139353   923 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
W20260709 13:13:41.142876   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 53643.4us
W20260709 13:13:41.144114   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.150434   925 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:41.149081   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:41.142956   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93195.8us
I20260709 13:13:41.153460   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:41.154192   853 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.151918   997 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:41.169474   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.169848   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.131328  3377 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:13:41.141772  3322 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:41.176638   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.182458   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:41.182837  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 97587.9us
I20260709 13:13:41.151381  3455 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:41.187139   921 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:41.191234   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 133697us
I20260709 13:13:41.192325   926 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.161782  3096 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:41.194633   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.156778  3272 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:41.156911  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.197503   930 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:13:41.199945   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.202024   923 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:41.205613   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 290470us
I20260709 13:13:41.199826   930 pending_rounds.cc:85] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:13:41.215942   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.216847   924 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:41.217715   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.220044  3335 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:41.221346   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.223763  3516 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:41.228065   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.228482  3496 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:41.200724  3515 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.229095  3496 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.229214  3515 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.234773  3448 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:41.235270  3448 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.235656   749 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 3 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:41.237911  3097 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:41.238721  3448 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.240137  3448 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.240419   922 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.242774  3496 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:41.201076  3272 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.244124  3272 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:41.156610  3498 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:13:41.187714  3455 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:41.259934   927 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:41.198912  3457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.005s	sys 0.018s
I20260709 13:13:41.250707   928 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:41.207896   930 pending_rounds.cc:107] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:13:41.263898   930 raft_consensus.cc:2889] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:41.264755   929 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:41.279152   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:41.268479  3457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.005s	sys 0.019s
W20260709 13:13:41.286942   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.263067  3096 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.263324  3455 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.288275   930 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:41.288841   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 183012us
I20260709 13:13:41.311528   749 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:41.317265   921 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:41.320304   922 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:41.327428  3479 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:41.327996  3479 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.335009  3515 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:41.336262  1596 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:41.337911  3096 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.340725   753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 225) took 9629 ms. Trace:
W20260709 13:13:41.341851  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.343956  3401 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:41.344607  3401 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.347520  3516 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.348189  1600 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:41.349611  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 463) took 1103 ms. Trace:
I20260709 13:13:41.349990  1600 rpcz_store.cc:276] 0709 13:13:40.256002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.257317 (+  1315us) service_pool.cc:224] Handling call
0709 13:13:40.257818 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.348687 (+1090869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.349301 (+   614us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400138dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:41.349517 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506546944}
I20260709 13:13:41.351716  3097 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:41.352914  3097 consensus_queue.cc:1243] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:41.353581  3498 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:41.354003  3498 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.355012  3498 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.356222  3498 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:41.360023   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.360777   930 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:41.361709   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.363029  3377 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:41.364028   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:41.364254  3322 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:41.364777   848 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:41.365581  3483 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:41.366031  3483 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.369024   921 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:41.320066   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:41.371129   923 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:41.372696   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:41.373618   926 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:41.378067  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.367934  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.288013  3187 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:13:41.380673   927 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:41.374083   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:41.390137   995 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.311452  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:41.392225  1001 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:41.393257  3305 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:13:41.394212  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.395975  1597 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:41.341189   753 rpcz_store.cc:276] 0709 13:13:31.710814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.711080 (+   266us) service_pool.cc:224] Handling call
0709 13:13:41.340493 (+9629413us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:36.671836 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:37.089306 (+417470us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:37.098008 (+  8702us) write_op.cc:435] Acquired schema lock
0709 13:13:37.101412 (+  3404us) tablet.cc:662] Decoding operations
0709 13:13:37.110532 (+  9120us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:37.110598 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:37.110630 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:13:37.196356 (+ 85726us) tablet.cc:701] Row locks acquired
0709 13:13:37.196385 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:13:37.236279 (+ 39894us) write_op.cc:270] Start()
0709 13:13:37.236412 (+   133us) write_op.cc:276] Timestamp: P: 1783602817236212 usec, L: 0
0709 13:13:37.236436 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:13:37.237067 (+   631us) log.cc:844] Serialized 5150 byte log entry
0709 13:13:37.473744 (+236677us) op_driver.cc:464] REPLICATION: finished
0709 13:13:37.973311 (+499567us) write_op.cc:301] APPLY: starting
0709 13:13:37.980171 (+  6860us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:38.002489 (+ 22318us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:38.002670 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:38.144588 (+141918us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:38.184467 (+ 39879us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:13:38.187194 (+  2727us) write_op.cc:312] APPLY: finished
0709 13:13:38.270912 (+ 83718us) log.cc:844] Serialized 95 byte log entry
0709 13:13:38.271092 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:38.271228 (+   136us) write_op.cc:454] Released schema lock
0709 13:13:38.271276 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:38.271662 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:38.271689 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14053,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":441633,"apply.run_cpu_time_us":5826,"apply.run_wall_time_us":329955,"num_ops":13,"prepare.queue_time_us":428638,"prepare.run_cpu_time_us":11585,"prepare.run_wall_time_us":718006,"raft.queue_time_us":93738,"raft.run_cpu_time_us":35814,"raft.run_wall_time_us":127195,"replication_time_us":237109,"thread_start_us":328829,"threads_started":4,"wal-append.queue_time_us":135787,"wal-append.run_cpu_time_us":22769,"wal-append.run_wall_time_us":1898714}]]}
I20260709 13:13:41.406519  3377 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:41.406603  3482 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.407163  3377 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.407560  3377 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.372310  3338 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.411994  2959 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.372340  3516 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.414656  2959 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:41.416635  3516 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:41.418656   999 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:41.420392  3401 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:41.421002  3401 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:41.422631  3428 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:41.402925  3272 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.386366  3483 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.423250  3428 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.423839  3483 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.424998  3272 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:41.424451  3428 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.415382  2959 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.426411  3428 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.392128   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.421052   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.428972   929 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:41.434343  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:41.436124   853 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
W20260709 13:13:41.435542   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 269074us
I20260709 13:13:41.455742   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:41.463955   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.435685  3404 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.001s
I20260709 13:13:41.469358   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.469403  1600 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.432891  3097 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:41.462424  1592 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:41.463207  3331 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.463755  3516 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.501714   927 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:41.500012   999 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:41.489002  3111 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:41.470485  3404 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5.
I20260709 13:13:41.521873   929 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:41.521571   995 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:41.524547  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:41.503165  3361 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:41.531338   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:41.539304   894 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:41.542659  1564 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:41.543830  3399 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.545442  3399 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:41.505461  2959 consensus_queue.cc:1059] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:13:41.525502  3331 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.512734  3429 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:13:41.505693  3423 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:13:41.545542  1564 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:41.559158   921 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:13:41.567049   921 pending_rounds.cc:85] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 6
I20260709 13:13:41.567286   921 pending_rounds.cc:107] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:13:41.567504   921 raft_consensus.cc:2889] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:41.565722   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 198578us
I20260709 13:13:41.521145  3097 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.518409  3516 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.551595   894 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:41.561561  3386 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:41.534109  3272 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:13:41.572412  3272 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:41.572872  3272 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.573971  3272 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.589259  3214 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:41.589967  3214 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:41.592236  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.593300  3377 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:41.596283  3523 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:41.561722  1597 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:13:41.604063  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58578us
I20260709 13:13:41.605547  3307 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:41.606122  3307 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:41.607028  3307 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.610271  3448 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:41.610693  3448 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.611392  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:41.607178  3479 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.613814  3457 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:41.614035  3448 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.614485  3457 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:41.616074  3457 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.617542  3448 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:41.620739  3515 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:41.621762  3187 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:41.622174  3187 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.623229  3187 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.624156  3187 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.595590  3377 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:41.550139  3519 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.646875   853 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:41.633332   996 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:41.554761  3427 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:41.575453  3272 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:41.663041  1600 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:13:41.664527   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 113142us
I20260709 13:13:41.678479   853 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:41.680600  3516 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:41.682355  3305 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:41.682567  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:41.683283  1597 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:41.684615   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:41.696075   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.718353   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52485.7us
I20260709 13:13:41.591092  3214 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:41.722566   997 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.548s	user 0.005s	sys 0.004s
I20260709 13:13:41.723006   997 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
W20260709 13:13:41.686447   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 407477us
I20260709 13:13:41.660919  3335 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:41.699031   895 connection.cc:256] RPC call timeout handler was delayed by -1.6944772459999058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:13:41.732336   895 connection.cc:256] RPC call timeout handler was delayed by -1.0407197109998378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
I20260709 13:13:41.685258  3482 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:41.750208   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.752067   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 67538.2us
I20260709 13:13:41.693336  3097 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:41.755870   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.757722   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.758698  3448 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:13:41.759060   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.758785   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.759995  3448 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.704923  2959 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:13:41.763181   929 raft_consensus.cc:2410] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:41.712173  3428 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:41.678392  3498 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:13:41.686421  3456 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:13:41.764937   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.726997  3447 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:13:41.819190   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.833736   929 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:41.749773  3361 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.721120  3305 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:13:41.724380  3427 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:41.651533  3519 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.761286  3483 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.753966  3111 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.754503  3404 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:41.829627   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:41.767688  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.693841  3331 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.832059   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.836297   969 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:41.726706  3516 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:13:41.836958   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:41.819185  3335 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.860543   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.863482   930 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:41.873827   923 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:41.837839  3214 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:41.874660   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46299.7us
I20260709 13:13:41.829197  3096 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.877403   927 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:13:41.764322  3448 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:41.877588   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.823552  3455 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.881464   929 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:41.878760   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.881434   924 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:41.897439   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 145858us
I20260709 13:13:41.830256  3496 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:13:41.900892   927 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:13:41.903215   923 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:41.903705   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:41.908097   930 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:41.859087  3498 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:41.880306  3482 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.893263  3386 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:41.879408  3483 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:41.918516   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.919649  3479 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.921970   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:41.926704  3516 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:41.927248  3516 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:41.928213   930 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:41.925176   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:41.875248  3404 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.880546  3335 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.930356   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.878057  3537 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:13:41.960708   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60072.2us
I20260709 13:13:41.913969  3428 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:41.960122  1567 leader_election.cc:336] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.605s (SENT)
I20260709 13:13:41.961787   923 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:41.921160  3423 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:41.921603  3482 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:41.922979  3498 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.967150   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.970985 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:41.975821   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 147637us
I20260709 13:13:41.913136  3524 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
W20260709 13:13:41.983480   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65730.7us
I20260709 13:13:41.962850  1597 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:41.906538  3096 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:41.966446  3377 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.984794   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:41.993923   923 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:41.992496   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.998380  3543 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:41.998682  3455 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.005584   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:42.006021   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:42.006426   927 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:42.007089   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:42.007431  3531 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:42.020044  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 131488us
I20260709 13:13:41.963937  3386 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.977169  3428 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:41.979660  3423 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:42.005667  3541 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:42.034729  3498 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:42.030609  2959 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:42.032526  3322 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:13:42.080796   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:42.079895   967 leader_election.cc:336] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.881s (SENT)
W20260709 13:13:42.117479  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.969311  3361 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.127882   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:42.128752   849 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:41.976604  3111 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.135509   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:42.141613   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.132758   821 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:42.114497  3364 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:42.081907  2959 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.152281  2959 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:42.129782  3533 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 13:13:42.143373  3531 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:42.149180  3550 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:42.149952  3539 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:13:42.155097   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:42.211565   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.232885   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:42.241894   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 162053us
I20260709 13:13:42.082029  3516 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.245052   927 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:42.248224   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:42.248854   930 raft_consensus.cc:2376] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6. Re-sending same reply.
I20260709 13:13:42.162272  3550 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:42.111318  3483 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:42.226231  3543 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:13:42.262584   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.267534  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.271129   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 262825us
W20260709 13:13:42.285789   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.293642   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.294154   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.294425   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.295651  3551 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:13:42.238288  3519 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.307674  3535 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:42.311686   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:42.160362  3482 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.323563   999 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:42.295727  3534 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.335s	user 0.001s	sys 0.000s
W20260709 13:13:42.329874  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 295482us
I20260709 13:13:42.330054   927 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:42.305871  3335 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:42.308950  3557 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:13:42.342604   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:42.343464   930 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:42.326906  3322 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:42.279729  3428 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:42.347486  1567 leader_election.cc:336] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.849s (SENT)
W20260709 13:13:42.349153   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53294.8us
W20260709 13:13:42.344008  3272 consensus_meta.cc:287] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.001s
I20260709 13:13:42.331303  3535 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:42.360867  3559 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:13:42.364737  3558 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:13:42.305027  3423 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:42.365693  3549 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:13:42.366921  3535 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.250294  3386 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:42.368693   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:42.333096  3377 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:42.372141  3536 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:42.373183  3550 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.374508  3570 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:42.298411  3498 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.388574   927 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:42.389989   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:42.391134   927 raft_consensus.cc:2410] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:42.375048  3570 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:42.408690   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:42.417275   930 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:42.416038  3533 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:13:42.488186  3377 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.499912  3570 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.355748  3531 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.623908  3526 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.624751  3526 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:42.527782  3479 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.598s	user 0.003s	sys 0.007s
W20260709 13:13:42.685060  3214 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
I20260709 13:13:42.695576   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1693) took 1340 ms. Trace:
W20260709 13:13:42.696008   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 583781us
I20260709 13:13:42.697681   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 293) took 1202 ms. Trace:
I20260709 13:13:42.697540   922 rpcz_store.cc:276] 0709 13:13:41.354858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.371364 (+ 16506us) service_pool.cc:224] Handling call
0709 13:13:41.375948 (+  4584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.388707 (+ 12759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.388739 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.462646 (+ 73907us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.649900 (+187254us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:41.649934 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.649997 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.406641 (+756644us) raft_consensus.cc:1685] finished
0709 13:13:42.406659 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.695500 (+288841us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":106408,"threads_started":1,"wal-append.queue_time_us":266555}
I20260709 13:13:42.699842   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:42.698144   921 rpcz_store.cc:276] 0709 13:13:41.494962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.502287 (+  7325us) service_pool.cc:224] Handling call
0709 13:13:41.525373 (+ 23086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.635447 (+110074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.697419 (+1061972us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54001f0754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:42.697617 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57394560}
I20260709 13:13:42.705343   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:42.708956   927 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:42.709824   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:42.710391   930 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
W20260709 13:13:42.711361  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 368845us
I20260709 13:13:42.711396   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:13:42.728140  2959 consensus_meta.cc:287] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.003s
I20260709 13:13:42.728825  2959 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.714221  3483 raft_consensus.cc:2764] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:42.774513  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:42.788664   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44640.7us
I20260709 13:13:42.779788  3526 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:42.802716  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.826148  3583 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:42.839924  3583 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:42.841612  3564 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.461s	user 0.008s	sys 0.004s
I20260709 13:13:42.842294  3583 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:42.845228   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.960s (SENT) [suppressed 34 similar messages]
W20260709 13:13:42.885753   967 leader_election.cc:336] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.548s (SENT)
I20260709 13:13:42.886256   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:42.886289  3515 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.889441  3479 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:42.904350   967 leader_election.cc:336] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.580s (SENT)
I20260709 13:13:42.904727   967 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:42.896020  3515 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:42.909332  3479 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:42.921322   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:42.923358  3404 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.936988   995 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:42.937438   995 pending_rounds.cc:85] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 2
I20260709 13:13:42.937736   995 pending_rounds.cc:107] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:42.938028   995 raft_consensus.cc:2889] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:42.939880  3515 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.932890  3404 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.947085   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.947335   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 298) took 1184 ms. Trace:
I20260709 13:13:42.948463   926 rpcz_store.cc:276] 0709 13:13:41.763496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.823743 (+ 60247us) service_pool.cc:224] Handling call
0709 13:13:41.824191 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.824665 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.824706 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.829815 (+  5109us) log.cc:844] Serialized 26 byte log entry
0709 13:13:42.842815 (+1013000us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b18003f9400. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:13:42.884377 (+ 41562us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:42.884421 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.884512 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.944662 (+ 60150us) raft_consensus.cc:1685] finished
0709 13:13:42.944690 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.947263 (+  2573us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23154,"raft.queue_time_us":584,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":2741,"spinlock_wait_cycles":4724096,"thread_start_us":41315,"threads_started":1,"wal-append.queue_time_us":1056062}
I20260709 13:13:42.950527  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:42.952299   995 raft_consensus.cc:2376] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3. Re-sending same reply.
W20260709 13:13:42.954167   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:42.954602   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.955530   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:42.945858  3588 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:13:42.958103  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:42.958927  1601 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:42.962013   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:42.962610   997 raft_consensus.cc:2376] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5. Re-sending same reply.
I20260709 13:13:42.964713   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:42.965363   999 raft_consensus.cc:2410] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:42.966336   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.959071  3552 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.960858  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:42.967697  1592 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:42.967519  3552 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.963637  3097 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.968945  3097 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.972245   997 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:13:42.972644  3097 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:42.975672  1001 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:42.976528   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 299) took 1125 ms. Trace:
I20260709 13:13:42.976889   929 rpcz_store.cc:276] 0709 13:13:41.850843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.851033 (+   190us) service_pool.cc:224] Handling call
0709 13:13:41.851475 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.902711 (+ 51236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.976306 (+1073595us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b540017a554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:42.976473 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68144512}
I20260709 13:13:42.977641   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.977829  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.978420   998 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:42.961184  3479 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.981283   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:42.981912   999 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:42.988593  1002 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:42.987748  3479 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:42.970840  3552 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:42.952800  3404 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:42.994711  3479 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.995242  1001 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:42.964088  3515 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.996819  3404 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:42.996768  3515 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:42.998971  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.999536   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:42.999917  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.000171   998 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.000320   999 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:43.000730  1002 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:13:42.983116  3591 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:13:43.014562  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:43.018540   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.036161   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:43.039163  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 67440.3us
I20260709 13:13:43.039865  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.043663  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.044306  1601 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:43.047854  3515 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.063213   997 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:43.068248  3386 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.069339  3526 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:43.072518   998 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:43.056430  3493 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:13:43.074821  3516 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.056576  3307 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:43.075289  3531 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:43.077476  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.079137  3483 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.080096   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:43.108786   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:43.080821   967 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.616s (SENT)
W20260709 13:13:43.088284   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.099363   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.084959  3482 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:43.110558  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:43.113067   967 leader_election.cc:336] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.728s (SENT)
I20260709 13:13:43.113596   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.100066   853 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:43.115056  3404 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:43.116227  3404 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:43.116438  3479 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:43.116683  3404 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:43.115979   967 leader_election.cc:336] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.775s (SENT)
I20260709 13:13:43.117904  3096 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:43.119560   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.121927   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.122016   998 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:43.121412  3097 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:43.126587   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.127156   853 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.102605  3214 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:43.106846  3563 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.125824  3097 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.123013  3479 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.119189  3531 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.136119   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.135576   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.135816   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.140832   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.142001   852 raft_consensus.cc:2410] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:43.099159  3386 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.123219  3096 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.163374   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:43.135383  3563 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.166261  3096 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.122179  3483 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.134328  3377 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.168221   848 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:43.176234   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.177076   997 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:43.177469   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.178174   852 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.178687   851 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:43.178768   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54970.3us
I20260709 13:13:43.123728  3482 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.180076   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.181007   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.181582   850 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.152879  3479 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.188683  3479 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.191444  3097 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.176100  3377 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.153847  3214 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.194006  3531 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.195432  3096 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.195545  3214 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.196346  3482 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.197196  3483 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.201731   846 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:43.202392  3361 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.184731  3563 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.203011  3361 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.205164  3404 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.207229  3404 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.208474  3361 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.207295  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.212081   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.217859  3386 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.220283   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.221058   851 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:43.186687   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:43.224473   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:43.233888   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.234694   851 raft_consensus.cc:2368] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:13:43.237895   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.238111   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.238623   854 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.238976   852 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:43.240147   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.245440   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.246090   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:43.248178   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.194031  3377 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.266263  3552 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:43.268762   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.236289  3596 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:43.244764  3097 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:43.270473   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1581) took 1368 ms. Trace:
I20260709 13:13:43.273072   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.277595   924 rpcz_store.cc:276] 0709 13:13:41.901607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.907336 (+  5729us) service_pool.cc:224] Handling call
0709 13:13:43.270200 (+1362864us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54001b0654. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:43.270393 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073549824}
I20260709 13:13:43.278227   847 raft_consensus.cc:2410] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:43.271083   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:43.279767   854 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:13:43.299202   854 pending_rounds.cc:85] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 2
I20260709 13:13:43.234259  3404 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:43.280069   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:43.269919  3599 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:13:43.304194   997 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.300129   854 pending_rounds.cc:107] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:13:43.305634   854 raft_consensus.cc:2889] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:43.276360  3280 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:43.274050  3596 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.309433  3280 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:43.309649  3596 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.309643  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.279340  3341 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:43.326759   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.329608   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.311501  1000 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.312059   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:43.310353  3404 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:13:43.310420  3280 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.313050   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.313989   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.315249   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.316305   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.335230   853 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:43.319103   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.322708   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.335626   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.330674   851 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.336485   997 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:43.268267  3552 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.337044   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.339116   851 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.269765  3607 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:13:43.304003  3377 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.362788  3377 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.363107   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.363997  3607 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:43.364364  3607 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.364686  3607 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.364853  3377 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.340734   850 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:43.334398  3401 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.343923   846 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.345036   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.366360  3401 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.367085   853 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:43.339296  3404 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:43.345798  3280 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.368455  3401 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.325875  3341 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:43.369011  3552 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.361130   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.355082   854 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:43.355337  2959 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.356037   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:43.358453   924 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:43.372561   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:43.360349  3111 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:43.338152  3214 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:13:43.339843  3479 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:43.336067   999 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.351923   852 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:43.359184   927 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:43.383752  3361 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:43.384207  3361 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:43.384584  3361 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.384625  3479 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.385133  3479 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:43.385416  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.388267   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.388880   999 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.392096  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.394207  1000 raft_consensus.cc:2368] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6.
I20260709 13:13:43.393996   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.395100   847 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:43.397727  3596 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.398684  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.375970  3097 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:43.400177  3596 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.400962  3097 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.393882  3609 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:43.404181  3609 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.404517  3609 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:43.407804   995 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:43.412603   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.417323  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.418041   851 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:43.417752  3280 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:43.420744  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.421401  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
W20260709 13:13:43.423763   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:43.424309   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:43.430420  3544 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.450s	user 0.012s	sys 0.000s
I20260709 13:13:43.431787  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.433045  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.433758  1000 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:43.434567  3608 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:13:43.435281   928 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:43.435717   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.436702  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:43.436784   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 296) took 1759 ms. Trace:
I20260709 13:13:43.437003   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:43.437565   927 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:13:43.437278   925 rpcz_store.cc:276] 0709 13:13:41.677085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.677238 (+   153us) service_pool.cc:224] Handling call
0709 13:13:41.681728 (+  4490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.693730 (+ 12002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.693766 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.694238 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.733996 (+ 39758us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:41.734046 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.713662 (+979616us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b54004448d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:42.713695 (+    33us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.430092 (+716397us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b24002b36b8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff8d89c4 00007f0cff8d6acb 00007f0cffa91dc9 00007f0cffa91a55 00007f0cffa9109d 00007f0d026c310e 00007f0d026da88c 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221
0709 13:13:43.434933 (+  4841us) raft_consensus.cc:1685] finished
0709 13:13:43.434957 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.436719 (+  1762us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36188160,"thread_start_us":39312,"threads_started":1,"wal-append.queue_time_us":220814}
W20260709 13:13:43.440512   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 106507us
I20260709 13:13:43.415548  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 498) took 1531 ms. Trace:
W20260709 13:13:43.447201   967 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.849s (SENT)
I20260709 13:13:43.447366   927 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:43.452647   999 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:43.452394   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.453434   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.454135   999 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.449990  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.456049   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.457629   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.462121   997 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:43.462266   999 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:43.440100  3596 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.472056   925 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:43.473253  3377 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:43.474915  1595 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:43.469101  3214 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.478047  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.478124  3214 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.480824  3214 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.483174  3596 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.485266  3596 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.443632  1596 rpcz_store.cc:276] 0709 13:13:41.880917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.893156 (+ 12239us) service_pool.cc:224] Handling call
0709 13:13:41.905843 (+ 12687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.971298 (+ 65455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.975953 (+  4655us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.217781 (+1241828us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b2c000e3668. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:13:43.218290 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.313472 (+ 95182us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:43.313543 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.313641 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.359903 (+ 46262us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b48002124d8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:43.391519 (+ 31616us) raft_consensus.cc:1685] finished
0709 13:13:43.392986 (+  1467us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.412226 (+ 19240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2360,"raft.queue_time_us":5160,"raft.run_cpu_time_us":10440,"raft.run_wall_time_us":1390469,"spinlock_wait_cycles":14491904,"thread_start_us":63062,"threads_started":1,"wal-append.queue_time_us":158320}
I20260709 13:13:43.486737   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1573) took 1663 ms. Trace:
I20260709 13:13:43.487130   928 rpcz_store.cc:276] 0709 13:13:41.823913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.828481 (+  4568us) service_pool.cc:224] Handling call
0709 13:13:43.435107 (+1606626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.435715 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.486562 (+ 50847us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54004448d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:43.486684 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3671708416}
I20260709 13:13:43.488142  1599 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:13:43.494196   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47042.3us
W20260709 13:13:43.494856   967 leader_election.cc:336] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.858s (SENT)
W20260709 13:13:43.497267   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56073.8us
I20260709 13:13:43.501255  1601 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:43.503947  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:43.508395   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.509012   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.509593   854 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.511130   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.518301   847 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:43.520445  1593 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
W20260709 13:13:43.521890   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.522570  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.523957   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.523828   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:43.534071   850 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:43.534992  1601 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:43.534809  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.547719   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.548378   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:43.548036   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.549605   847 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:43.547675   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 49200.3us
I20260709 13:13:43.550714   854 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
W20260709 13:13:43.550812   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55986.1us
I20260709 13:13:43.501519  3483 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.462774  3361 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.517984  2959 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.547475   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:43.518123  3377 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.517795  3214 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.535809   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:43.556097   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.550652   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:43.561180   852 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:43.561553   928 raft_consensus.cc:2376] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3. Re-sending same reply.
I20260709 13:13:43.558557  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:43.559929   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.557722  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.568548  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.569128   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.570088  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.547391  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.576058  3361 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.571633   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:43.576934   850 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:43.578096   850 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:43.578547  3607 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.567818  3515 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:43.572095  3477 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:43.580646   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.581791   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.582861   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.571568  3214 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.564867  2959 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.584424   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.583460   928 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:43.548797  3613 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:43.556221  3583 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:43.706357   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:43.707304   967 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.942s (SENT)
I20260709 13:13:43.567720  3610 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.571686  3600 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:43.585299  3584 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:43.718518   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.716714  3364 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:43.722313  3097 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.561424  3483 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.720116   921 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:13:43.620307  3604 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:13:43.703665  3482 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:43.571698  3377 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.710866  3515 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:43.585638  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:43.584654  3531 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.135s
I20260709 13:13:43.585426  3611 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.161s
I20260709 13:13:43.856012   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.735816  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:43.857286  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 271702us
I20260709 13:13:43.849550  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:43.857940   921 pending_rounds.cc:85] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 2
I20260709 13:13:43.871738   921 pending_rounds.cc:107] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:43.584527  3609 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:43.854434   749 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 118) took 12765 ms (client timeout 15000 ms). Trace:
W20260709 13:13:43.870703  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 300659us
I20260709 13:13:43.859103   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.876257   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.877161   998 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:43.738001   995 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:13:43.879652   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.872109   921 raft_consensus.cc:2889] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:43.872728  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.883064   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:43.883651   852 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.874329   997 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
W20260709 13:13:43.873284   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 304217us
I20260709 13:13:43.886114   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:43.887163   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:43.794798  3618 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:13:43.889703   851 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:43.892709   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.882041  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:43.820616  3623 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:43.899595   928 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.901223   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.902150   928 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:43.894492   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.895995   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:43.902936   998 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:43.782691  3307 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:43.869529  3364 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.846370  3607 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:43.760156  3563 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:43.904865   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:43.905408   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.907232   997 raft_consensus.cc:2368] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:13:43.790004  3214 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:43.723758  3479 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.912425   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:43.791563  3427 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:43.885114  1596 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.872217  3526 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:43.915714   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:43.872913  3613 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.914065   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:43.872745   749 rpcz_store.cc:269] 0709 13:13:31.088878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.100281 (+ 11403us) service_pool.cc:224] Handling call
0709 13:13:43.854349 (+12754068us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:41.593711 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:41.594355 (+   644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:41.594406 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:13:41.594425 (+    19us) tablet.cc:662] Decoding operations
0709 13:13:41.595039 (+   614us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:41.595128 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:41.595151 (+    23us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:13:41.649298 (+ 54147us) tablet.cc:701] Row locks acquired
0709 13:13:41.684090 (+ 34792us) write_op.cc:260] PREPARE: finished
0709 13:13:41.699428 (+ 15338us) write_op.cc:270] Start()
0709 13:13:41.699675 (+   247us) write_op.cc:276] Timestamp: P: 1783602821699340 usec, L: 0
0709 13:13:41.699710 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:13:41.700590 (+   880us) log.cc:844] Serialized 1674 byte log entry
0709 13:13:42.152528 (+451938us) op_driver.cc:464] REPLICATION: finished
0709 13:13:42.375968 (+223440us) write_op.cc:301] APPLY: starting
0709 13:13:42.376073 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:42.376240 (+   167us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:42.376374 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:42.376977 (+   603us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:42.377246 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:13:42.377304 (+    58us) write_op.cc:312] APPLY: finished
0709 13:13:42.377640 (+   336us) log.cc:844] Serialized 41 byte log entry
0709 13:13:42.377824 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:42.392178 (+ 14354us) write_op.cc:454] Released schema lock
0709 13:13:42.406371 (+ 14193us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:42.408419 (+  2048us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:42.408456 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2772176,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":223107,"apply.run_cpu_time_us":2585,"apply.run_wall_time_us":32416,"num_ops":4,"prepare.queue_time_us":251951,"prepare.run_cpu_time_us":13112,"prepare.run_wall_time_us":235206,"raft.queue_time_us":114446,"raft.run_cpu_time_us":34463,"raft.run_wall_time_us":260550,"replication_time_us":420051,"thread_start_us":449815,"threads_started":4,"wal-append.queue_time_us":203961,"wal-append.run_cpu_time_us":12041,"wal-append.run_wall_time_us":1578450}]]}
I20260709 13:13:43.872073  3519 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:13:43.931090  2959 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:43.932454  3626 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:43.932235  3341 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.933372   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:43.971896   929 raft_consensus.cc:2376] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6. Re-sending same reply.
I20260709 13:13:43.935858   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:43.938238   995 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:43.944830   854 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:43.973011  1000 raft_consensus.cc:2368] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
I20260709 13:13:43.897534  3583 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:43.884621  3361 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:13:43.967401   852 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:43.970767   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.981624   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:43.896355  3482 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:13:43.883514  3610 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.988281   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.991672   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.992271   921 raft_consensus.cc:2376] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9. Re-sending same reply.
W20260709 13:13:43.973069  3617 log.cc:927] Time spent T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:43.884711  3097 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:43.941784  3214 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:44.001008   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.006611   928 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:43.969377  3516 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:44.005378  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.913529  3563 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:43.966662  3526 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.007710   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.009112   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.015542   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.015926   929 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:44.016310   921 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:43.970201  3613 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.015556   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.019556   928 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:44.020768   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:43.967557  3427 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.007159  3626 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:43.999886  2959 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:44.136545   924 raft_consensus.cc:2410] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:44.045058   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.041649   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:44.138136   854 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:13:44.130514   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 148944us
I20260709 13:13:44.033011   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.141959   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.018913  3526 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:44.145726   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.031528  3626 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:44.112686  3623 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:44.141250   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.112588  3447 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:44.156621  3447 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:44.045087   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.151774   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 216008us
W20260709 13:13:44.152643   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 445341us
I20260709 13:13:44.158962  3447 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.142539  3341 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:44.161597   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.147884   818 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:44.163753   818 leader_election.cc:403] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:44.164384  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:44.164772   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.007388  3515 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.165926   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.167054  3516 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.167522  3516 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.154613   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.165535  2959 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:44.168720  3619 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:13:44.114399  3609 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.169814  3516 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:44.170305  3479 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:44.171969   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.173380  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:44.177296  3531 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.177950  3626 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.179792  3630 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5.
I20260709 13:13:44.182813   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.183883   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:44.183965   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.185392   821 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.156962   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.187713  3630 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:44.187875   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.188422  3630 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.188524   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.188851  3630 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.189116   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.189947   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.190268   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.191009   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.191646   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.168740   922 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:44.189214  3630 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.191386   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.041798  3483 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.202033   929 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:44.209650   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.212545  3563 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.216414  3610 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.218006  3610 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:44.218688  3610 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.219739   928 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:44.230654   854 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:44.201048  3427 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.242352   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.245221   922 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:44.275760   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.276859   927 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:44.208940  3483 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:44.280669   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.281462   929 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:44.283645   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 137054us
W20260709 13:13:44.284272   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.284123   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.286599   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:44.287230   854 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:44.203042  3583 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.244691  3097 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:44.278671   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.301627   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.302426   927 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:44.302636   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.295064  3447 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:44.304023   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 118678us
I20260709 13:13:44.304792   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.305104   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.307240   928 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:44.309221   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.309443   928 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:44.310853   928 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:44.313910   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:44.318198   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.318719   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.319070   852 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:44.295145  3634 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.295614  3584 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:44.322257   929 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:44.334506   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1667) took 1214 ms. Trace:
I20260709 13:13:44.336174   854 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:44.334074   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.329483   927 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:44.338039   852 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:44.337556   854 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:44.339148   927 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:44.339733   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.342839   929 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:44.315838  3637 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:44.320200  3610 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:44.345386   895 leader_election.cc:336] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.547s (SENT)
I20260709 13:13:44.347990   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:44.352412  3622 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
W20260709 13:13:44.360451  3639 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:13:44.335346  3584 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:44.361771   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.363055  3584 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.346863  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1559) took 1335 ms. Trace:
I20260709 13:13:44.376088   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1756) took 1266 ms. Trace:
I20260709 13:13:44.376523   849 rpcz_store.cc:276] 0709 13:13:43.109970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.110108 (+   138us) service_pool.cc:224] Handling call
0709 13:13:43.111897 (+  1789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.194584 (+ 82687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.194614 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:43.194654 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:43.194685 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.375838 (+1181153us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b54002b9154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:44.375890 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.376016 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189705088}
I20260709 13:13:44.366026  1003 rpcz_store.cc:276] 0709 13:13:43.011808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.322944 (+1311136us) service_pool.cc:224] Handling call
0709 13:13:44.346371 (+ 23427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.346561 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.346586 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.346630 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:44.346662 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.346711 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.346812 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.385217   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.386505   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.345818  3214 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.387378  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:44.317245  3447 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.346963  3633 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 13:13:44.391216  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.399901   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:44.347071  3637 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.328470  3638 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
W20260709 13:13:44.398095   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 206509us
I20260709 13:13:44.405828   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.411815   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:44.407835   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:44.375428  3610 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.395583   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.363747  3341 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:13:44.335165  3634 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.415509  3429 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:44.412298   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.419641  3633 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.449935  3633 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:44.450491  3633 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.451530  3625 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.452090  3625 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.454977  3625 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.421976  3447 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.428166  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:44.456892  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:44.457444  1003 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:44.458551  1003 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:44.391849  3214 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:44.459817  3214 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.461853  3634 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.462425  3634 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:44.463060  3634 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:44.410858  3637 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:44.464109  1596 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:44.465091  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:44.430542   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.465727  1592 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:44.466566   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.467922   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:44.364557  3526 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.445078  3610 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.469053   818 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.469758   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:13:44.439029   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93877.1us
I20260709 13:13:44.472306   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:44.467753   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:44.443403  3364 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.504s	user 0.001s	sys 0.006s
I20260709 13:13:44.477272   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:44.478274  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.413583  3455 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:44.494055   853 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:13:44.494458   853 pending_rounds.cc:85] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:44.494370  3455 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:44.494768   853 pending_rounds.cc:107] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:44.495040   853 raft_consensus.cc:2889] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:44.495379  3455 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.482108  3423 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.499336  3423 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.501447  3423 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.468647  3648 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.486112  3097 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.487340  3537 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:44.488998  3633 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.489439  3610 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.416589  3609 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:44.470660  3111 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:44.493461  3526 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.410696  3613 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.508777  3648 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.509414   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:44.525871   849 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:44.573179   849 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:44.503134  3377 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.556052  3111 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.504559  3636 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:13:44.510103   895 leader_election.cc:336] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.709s (SENT)
I20260709 13:13:44.509851  3097 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.512079  2959 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.558642  3531 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:13:44.509533  3429 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:13:44.508544  3364 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:44.570117   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 162362us
I20260709 13:13:44.525874  3519 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:13:44.559405  3648 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.589196  3613 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.593068  3214 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.579537  3636 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:44.596119  3214 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.596470  3636 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:44.596881  3427 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:44.597343  3427 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:44.597759  3427 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:44.578136   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1674) took 1005 ms. Trace:
I20260709 13:13:44.567574  3609 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:44.608479  3609 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.609081  3111 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.610671  3609 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:44.557132  3638 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
W20260709 13:13:44.577468   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68116.3us
I20260709 13:13:44.614542   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.616534   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.616725  3630 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.617134   924 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:44.618822  3630 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.622519   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.622929   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:44.623317   926 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:44.624662   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.625334   924 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:13:44.543943   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94551.5us
I20260709 13:13:44.574926  3423 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:44.586452   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80844.2us
I20260709 13:13:44.566454  3563 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:13:44.582320  3097 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.564342  3646 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 13:13:44.637823   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51812.4us
I20260709 13:13:44.629045   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.630761   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:44.633214   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 1783) took 1122 ms. Trace:
I20260709 13:13:44.641319   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:44.627244   999 rpcz_store.cc:276] 0709 13:13:43.572638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.489591 (+916953us) service_pool.cc:224] Handling call
0709 13:13:44.489889 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.490123 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:44.490405 (+   282us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.576421 (+ 86016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.576462 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.576500 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.576560 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.578038 (+  1478us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":290,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":19347,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":85605,"threads_started":1,"wal-append.queue_time_us":123888}
I20260709 13:13:44.643814  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 294) took 1667 ms. Trace:
I20260709 13:13:44.640408   851 rpcz_store.cc:276] 0709 13:13:43.510688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.523725 (+ 13037us) service_pool.cc:224] Handling call
0709 13:13:44.633067 (+1109342us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54000726d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:44.633164 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839414528}
I20260709 13:13:44.572804  3645 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:13:44.646896   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:44.649859   926 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.644268  1001 rpcz_store.cc:276] 0709 13:13:42.976765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.981053 (+  4288us) service_pool.cc:224] Handling call
0709 13:13:42.984443 (+  3390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:42.996573 (+ 12130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.636382 (+1639809us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54000884d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:44.643739 (+  7357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24371456}
I20260709 13:13:44.650182   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:44.651047   999 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:44.652731   999 pending_rounds.cc:85] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 5
I20260709 13:13:44.653196   999 pending_rounds.cc:107] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:13:44.653488   999 raft_consensus.cc:2889] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:44.660166   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:44.667945  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:44.668998  1001 raft_consensus.cc:2410] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:44.670328  1001 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:44.677551  3596 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:44.678045  3633 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:44.681878  3493 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:44.678160  3596 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:44.649231  3630 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.657166  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:44.633693  3423 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.655754  3638 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.695181  1001 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:44.703473  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:44.649133  3609 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:13:44.709885   848 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: [5.7-8.8]
I20260709 13:13:44.683815  3654 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.700525  3630 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.675067  3637 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:44.720427   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 56899.7us
I20260709 13:13:44.723378   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.724129   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.742060   851 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:13:44.742688   853 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:44.706561  3638 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.683219  3645 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:13:44.727567   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67514.3us
I20260709 13:13:44.716732  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:44.721611  3630 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.762683   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:44.754606   926 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:44.764047   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:44.764647   848 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.745468  1592 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:13:44.768157  3657 log.cc:927] Time spent T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:44.726562  3632 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:44.766263   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:44.774013   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 147285us
I20260709 13:13:44.785615   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:44.745710  3638 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:44.802103  3515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.000s	sys 0.019s
W20260709 13:13:44.802595  3515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.019s
I20260709 13:13:44.806296   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.806145  3493 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:44.807339  3625 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:44.806221  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:44.810330  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.811271  1592 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:44.820739   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:44.823025 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:44.790832  3482 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:44.838969   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53462.3us
I20260709 13:13:44.752055  3596 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.839713   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:44.844650  3656 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:13:44.807741  3625 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:44.849272   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:44.832070  3498 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:44.834795  3584 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:44.840674  3609 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.840334  3654 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:44.838470  2959 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:44.859716  3652 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:13:44.861205  3652 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.861663  3652 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.862044  3652 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:44.862135  3661 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:13:44.863245  3662 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:44.858835  3584 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:44.864048  3584 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:44.867944  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 310) took 1354 ms. Trace:
I20260709 13:13:44.868382  1003 rpcz_store.cc:276] 0709 13:13:43.513105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.459208 (+946103us) service_pool.cc:224] Handling call
0709 13:13:44.459516 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.471998 (+ 12482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.472023 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.475943 (+  3920us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.696477 (+220534us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:44.724679 (+ 28202us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.725060 (+   381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.725151 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.867688 (+142537us) raft_consensus.cc:1685] finished
0709 13:13:44.867714 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.867878 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29884,"raft.queue_time_us":157,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":8355,"spinlock_wait_cycles":81507072,"thread_start_us":136664,"threads_started":1,"wal-append.queue_time_us":219551}
I20260709 13:13:44.870728  1003 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:44.871560  3648 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.850576  3664 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.834561  3636 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.876168  3609 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.841821  3482 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:44.895363   895 connection.cc:256] RPC call timeout handler was delayed by -2.1716634140000224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30970000000000003s for a response
I20260709 13:13:44.843211  3610 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:44.885499  3664 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.884845  3654 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:44.885746  3483 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:44.905819   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 139622us
I20260709 13:13:44.858966  3515 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:13:44.907537   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:44.909325  3660 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:44.909549  3519 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.909866  3660 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.910117  3519 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:44.899592  3609 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.897738  3625 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.916025  3625 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:44.916450  3519 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.917615  3660 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.917891  3096 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:44.919821  3515 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.921536  3636 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:44.908385  3423 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.906704  3664 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.916803  3097 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:44.912609  3637 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:44.976889   895 connection.cc:256] RPC call timeout handler was delayed by -1.4710173420000956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
W20260709 13:13:44.978629   895 connection.cc:256] RPC call timeout handler was delayed by -1.4134173420000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 13:13:44.979231   895 connection.cc:256] RPC call timeout handler was delayed by -1.3945173420001993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
I20260709 13:13:44.938524  3515 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:44.976091  3665 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:13:44.936676  3096 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:45.004235  3584 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.046945  3096 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.048349  3096 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.049124  3584 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:44.985072  1001 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:13:44.980043   895 connection.cc:256] RPC call timeout handler was delayed by -1.3441173420001178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:13:45.053289   895 connection.cc:256] RPC call timeout handler was delayed by -1.2685173420002229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
W20260709 13:13:45.053627   895 connection.cc:256] RPC call timeout handler was delayed by -1.2586173420002069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:13:44.994700  3663 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:13:45.006256  3515 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.046952  3482 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.004865   848 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:44.923236  3427 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.066109  3427 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.066737  3652 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:45.068476  3674 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:13:45.099931  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 513) took 1367 ms. Trace:
I20260709 13:13:45.121333   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1679) took 1547 ms. Trace:
W20260709 13:13:45.121526  1000 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33108 (request call id 300) took 2079 ms (client timeout 2489 ms). Trace:
I20260709 13:13:45.120363  1601 rpcz_store.cc:276] 0709 13:13:43.732930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.733092 (+   162us) service_pool.cc:224] Handling call
0709 13:13:43.733425 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.939140 (+205715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.939186 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:43.939244 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:43.939290 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.099643 (+1160353us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54000e5bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:45.099695 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.099858 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472641920}
I20260709 13:13:44.977865  3660 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:45.121922  1000 rpcz_store.cc:269] 0709 13:13:43.041871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.434268 (+392397us) service_pool.cc:224] Handling call
0709 13:13:45.121324 (+1687056us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540022c4d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:45.121480 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191152000}
I20260709 13:13:45.121834   997 rpcz_store.cc:276] 0709 13:13:43.573669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.892488 (+318819us) service_pool.cc:224] Handling call
0709 13:13:45.121061 (+1228573us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b54001aa4d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:45.121259 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31670528}
W20260709 13:13:45.071035  3628 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.903s	user 0.003s	sys 0.000s
W20260709 13:13:45.133397  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.075456  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:45.129001  3482 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.053303  3630 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.137902  3482 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.138283  1592 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:45.139710  3630 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.139861  3482 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:45.150967   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 243494us
I20260709 13:13:45.152047   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.152369   895 leader_election.cc:400] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:13:45.152837   895 leader_election.cc:403] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:45.160126   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.162325   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.164873   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:45.167088   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.169538   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.171350   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:45.163771  3596 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:45.173183   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.174116  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.174324   852 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:45.175289  3672 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:45.175841  3672 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:45.176285  3672 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.984270  3654 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.135775  3660 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:45.178395   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 169208us
I20260709 13:13:45.180182   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:45.181871   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:45.186487   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.187494   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.189044   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.189693   852 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:45.190699   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.191401   850 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7.
I20260709 13:13:45.194795   821 leader_election.cc:400] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:13:45.198450  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1557) took 2185 ms. Trace:
I20260709 13:13:45.186148  3341 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.191529  3097 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:45.202840  1002 rpcz_store.cc:276] 0709 13:13:43.011219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.011362 (+   143us) service_pool.cc:224] Handling call
0709 13:13:43.052139 (+ 40777us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:43.052580 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.052609 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:44.679139 (+1626530us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b2c000e6738. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:13:44.679809 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 13:13:44.725304 (+ 45495us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.725335 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.725401 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.191072 (+465671us) raft_consensus.cc:1685] finished
0709 13:13:45.191093 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.196608 (+  5515us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2820,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":91,"spinlock_wait_cycles":82390400,"thread_start_us":45106,"threads_started":1,"wal-append.queue_time_us":109840}
I20260709 13:13:45.190801  3596 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:45.207991   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.208654  3678 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:13:45.208884  3684 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:45.209715  3427 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:45.210325  3614 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:45.211356  3679 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:45.212273   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1778) took 1703 ms. Trace:
I20260709 13:13:45.205849  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:45.229689   999 raft_consensus.cc:2410] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:45.226055   852 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:45.194796  3677 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:45.236748  3214 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:45.237221  3214 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:45.230672  3684 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.238217  3214 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.240219  1592 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.242043  3214 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.244458  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:45.245347  1600 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:13:45.245754  1600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 488) took 1859 ms (client timeout 1613 ms). Trace:
W20260709 13:13:45.246099  1600 rpcz_store.cc:269] 0709 13:13:43.386068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.387392 (+  1324us) service_pool.cc:224] Handling call
0709 13:13:44.655769 (+1268377us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:45.244315 (+588546us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800003c98. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ae9c 00007f0cffa1aa5b 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:45.245682 (+  1367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11402368}
I20260709 13:13:45.253334  3364 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:45.257091   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49161.5us
I20260709 13:13:45.258172  3677 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.260044   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.230176   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.229099   846 rpcz_store.cc:276] 0709 13:13:43.509224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.509969 (+   745us) service_pool.cc:224] Handling call
0709 13:13:43.551860 (+ 41891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.553899 (+  2039us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:43.584915 (+ 31016us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.941082 (+356167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.991328 (+ 50246us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.991477 (+   149us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:43.991518 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.991583 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.914845 (+923262us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:45.211973 (+297128us) raft_consensus.cc:1685] finished
0709 13:13:45.212068 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.212200 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17238,"prepare.queue_time_us":29995,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":35538,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":22131,"spinlock_wait_cycles":62212992,"thread_start_us":220635,"threads_started":1,"wal-append.queue_time_us":441269}
I20260709 13:13:45.232070  3679 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.232623  3629 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:45.235522  1566 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:45.261781   854 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:45.226778  3652 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.283205   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.284189   854 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:45.284930   852 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:45.289517   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:45.289996   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:45.294415   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.295117  3625 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:45.295748  3625 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:45.296109   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.297153  3677 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.296924  3625 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.297720  3677 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.299373  3673 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:45.300200  3677 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.300123  3673 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.303014  3673 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.289062  3429 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:45.316088   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.292088  3427 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.320708  3427 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:45.321242  3427 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:45.322206  3427 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.228174  3609 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:45.325228  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.328811   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.329521   853 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.332285   852 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:45.332592  3679 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.333791  3684 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.339756   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:45.342458   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.343739   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53861.9us
I20260709 13:13:45.344381   921 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:45.344766   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.345048   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:45.345877   997 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:45.333882  1592 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:45.340633   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:45.358559   853 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:45.360872   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.362003   853 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:13:45.368901  3626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.188s	user 0.007s	sys 0.015s
W20260709 13:13:45.369306  3626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.007s	sys 0.015s
I20260709 13:13:45.369410   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:45.369432  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 133975us
I20260709 13:13:45.370285  3519 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.371565   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.372411  3423 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.372812   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.373636  3423 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:45.374863  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:45.375882  3493 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:45.376459  3493 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:45.376737   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.377542  3493 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.382716   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.383831   854 raft_consensus.cc:2368] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:13:45.384416   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.386294   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.363215  3652 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:45.385648   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.389088   852 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:45.389209   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.390791   853 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:45.384893   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.392573   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.391884   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.346043  3479 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.346302  3679 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.346020  3684 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.395804   853 raft_consensus.cc:2368] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
I20260709 13:13:45.390411   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.397714   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.352612  3361 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:45.397771   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.387323  3652 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:45.401371  3652 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.401134  3684 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.283776  3629 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.412827   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.366981  3677 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:13:45.424355  3638 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.003s
I20260709 13:13:45.424580   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.405129  3479 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.429235   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.429925   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.437124   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.431039  3361 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:45.431691  1002 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:45.432500  3629 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.432732  3672 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.415834  3679 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.440923   853 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:45.422433   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.443266   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.427007  3515 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.403585  3684 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.455482   926 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:45.444103   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.471550   929 raft_consensus.cc:2376] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9. Re-sending same reply.
I20260709 13:13:45.459594   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.460407   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:45.463932   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.473852   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.476797   925 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:45.449618  3583 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:45.453871  3660 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:45.479390   967 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:45.480175   852 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:13:45.482678  1000 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:45.498786   852 pending_rounds.cc:85] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:45.484627   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.485630   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1562) took 2468 ms. Trace:
I20260709 13:13:45.499197   852 pending_rounds.cc:107] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:45.499482   852 raft_consensus.cc:2889] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:45.488166   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1725) took 1301 ms. Trace:
I20260709 13:13:45.501129   928 rpcz_store.cc:276] 0709 13:13:44.187266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.318159 (+130893us) service_pool.cc:224] Handling call
0709 13:13:44.490891 (+172732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.491090 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:44.491347 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.573975 (+ 82628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.574028 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.574072 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.574135 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.487915 (+913780us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400175050. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:45.488090 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8503,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":3147,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":4743,"spinlock_wait_cycles":396556160,"thread_start_us":82333,"threads_started":1,"wal-append.queue_time_us":196616}
I20260709 13:13:45.506162   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.506289   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.499379   996 rpcz_store.cc:276] 0709 13:13:43.016714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.460659 (+443945us) service_pool.cc:224] Handling call
0709 13:13:44.636523 (+1175864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.657800 (+ 21277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.657833 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.665287 (+  7454us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.801367 (+136080us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:44.801422 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.801515 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.808824 (+  7309us) raft_consensus.cc:1685] finished
0709 13:13:44.808853 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.485244 (+676391us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000884d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:45.485540 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20888,"raft.queue_time_us":20986,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":25995,"spinlock_wait_cycles":2627228416,"thread_start_us":83147,"threads_started":1,"wal-append.queue_time_us":99460}
I20260709 13:13:45.507076   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:45.503918   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:45.508371   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.506356   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:45.495559  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 203989us
I20260709 13:13:45.448997  3638 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.509740   925 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.484009  3515 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.476688  3626 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:45.512378   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:45.494712  3684 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:45.496243  3675 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.505604  3634 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:45.513891  3686 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.513761  3687 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:45.522517   847 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:45.524369  1601 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:13:45.523329   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:45.518441  3664 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 13:13:45.529377   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.514628  3482 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:45.530247   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.530853   996 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:45.540843   925 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:45.542187   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.505764  3673 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:13:45.501947  3583 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:45.521579  3097 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:45.554812  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:45.559537   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.557435   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.533048  3634 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:45.547971  3609 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.523938  3515 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:13:45.590958  3479 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.525625  3675 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.594012  3675 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.595498  3638 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.599785   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.601428   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.602010   927 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.565240   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:45.518931  3361 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.569823   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:45.609663   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.570418  3693 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:13:45.564149  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:45.573643  3681 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.379s	user 0.003s	sys 0.001s
I20260709 13:13:45.574067  3660 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:45.576079  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:45.577085   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:45.581527   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.633879   999 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:44.335179   998 rpcz_store.cc:276] 0709 13:13:43.120446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.922188 (+801742us) service_pool.cc:224] Handling call
0709 13:13:43.922592 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.934054 (+ 11462us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:44.020883 (+ 86829us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.334071 (+313188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.334213 (+   142us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.334271 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.334338 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.334451 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87233,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1988,"thread_start_us":198635,"threads_started":1}
I20260709 13:13:45.585811  3679 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.587399  3583 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:45.551412  3482 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:45.560251  3097 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:45.624701  3677 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:45.630541   927 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:45.631757   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.545662  3664 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.638370   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.727828   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:45.728369   998 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:45.621230  3672 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:45.568751  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.640882   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.729990   929 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:45.724346   925 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.730468   929 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:45.735140  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 166500us
I20260709 13:13:45.735970  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:45.736485   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:45.623548  3537 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:13:45.624873  3636 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 13:13:45.630322  3519 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 13:13:45.740254   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 130017us
I20260709 13:13:45.739297   929 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.747848   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:45.726136  3677 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:45.809342  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71992.3us
I20260709 13:13:45.810606   998 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:45.812268   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:45.813410   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50332.7us
I20260709 13:13:45.814245   929 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:45.815662   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.815346  3482 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.816319   999 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:45.817265   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.818415   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:45.825845   925 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:45.827672   999 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:45.829180   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:45.740892  3526 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:45.933386   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:45.810894  3677 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:45.839897  3214 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.003s
I20260709 13:13:45.943985   848 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:45.939167   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1736) took 1309 ms. Trace:
W20260709 13:13:45.939266   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 110153us
I20260709 13:13:45.938839  3483 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:45.954494   999 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:45.957006  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1576) took 1298 ms. Trace:
I20260709 13:13:45.957444  1003 rpcz_store.cc:276] 0709 13:13:44.658412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.871542 (+213130us) service_pool.cc:224] Handling call
0709 13:13:44.872385 (+   843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.872729 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.872758 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.873264 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.155551 (+282287us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:45.155589 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.155680 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.572834 (+417154us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:45.835134 (+262300us) raft_consensus.cc:1685] finished
0709 13:13:45.835164 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.956920 (+121756us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46835,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":3951360,"thread_start_us":281920,"threads_started":1,"wal-append.queue_time_us":312309}
I20260709 13:13:45.945740   924 rpcz_store.cc:276] 0709 13:13:44.629759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.629916 (+   157us) service_pool.cc:224] Handling call
0709 13:13:44.638007 (+  8091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.724621 (+ 86614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.724653 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.737948 (+ 13295us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.838724 (+100776us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.838776 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.938788 (+1100012us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54004466d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:45.938838 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.938883 (+    45us) raft_consensus.cc:1685] finished
0709 13:13:45.938897 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.939107 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1413,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2709,"spinlock_wait_cycles":190087552,"thread_start_us":66448,"threads_started":1,"wal-append.queue_time_us":103221,"wal-append.run_cpu_time_us":1887,"wal-append.run_wall_time_us":1063302}
I20260709 13:13:45.963402   998 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:45.964952   998 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:45.932832  3550 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:45.978762   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63983.4us
I20260709 13:13:45.982806   998 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:45.896059  3307 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:13:45.969015  3483 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:46.005729  3483 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.006577  3664 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:46.008082  3698 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:13:45.983544  3214 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.036010  3648 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.037423  1002 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:46.038655  1002 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:46.040295  3712 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:46.041253  1002 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:46.040807  3712 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:45.952521  3097 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:46.064895  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.069969   929 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:46.015251  3652 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 13:13:45.757869  3634 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.081454  1002 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:46.090464   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 342745us
I20260709 13:13:46.043402  3214 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.098603  1002 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:46.097137   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:46.080930  3652 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.105461  3652 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:46.108938  3704 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:13:46.110532  3697 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:13:46.113180  3700 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:13:46.114583  3708 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:13:46.115199   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:46.145377  3214 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.138429  3709 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:46.152326  3709 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:46.152722  3709 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.158383  3611 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:46.159127  3611 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.161720  3611 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.170164  3711 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.171097  3711 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:46.171742   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 47072.3us
I20260709 13:13:46.175177  3712 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.150982  3677 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.180562   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.190244   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:46.188958   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 61924.4us
I20260709 13:13:46.194478  3712 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:46.194554  3717 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:46.196172  3677 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.193051   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:46.190140  3638 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:46.201814  3638 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.202226   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.203516   895 leader_election.cc:400] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:13:46.198861   926 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:46.199759  3611 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:46.230665   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:46.176507  3711 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:46.240892   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.238587  3625 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:46.243084  3214 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:46.310310  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.248838  3611 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:46.276219  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:46.240567  3712 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:13:46.320842  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:46.320909   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.238291  3717 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:13:46.321064  3638 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.328986  1597 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:46.320948  3718 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.315388  3625 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.313921  3111 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:13:46.329537  3722 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.328168  3710 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:13:46.377349  3710 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.382166  1004 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:46.382673  3607 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.345394  3711 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.388382  3607 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.390359  3611 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:46.395733  1601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.003s	sys 0.024s
W20260709 13:13:46.396275  1601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.003s	sys 0.024s
W20260709 13:13:46.397850  3660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.018s	sys 0.013s
W20260709 13:13:46.398591  3519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.015s	sys 0.016s
I20260709 13:13:46.367571  3712 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:13:46.397693  3423 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:46.369942  3721 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:13:46.419315   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:46.466070  3519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.015s	sys 0.017s
W20260709 13:13:46.423821  3660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.019s	sys 0.013s
I20260709 13:13:46.477684  3721 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:46.417574  3727 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:46.476807  3712 consensus_queue.cc:1243] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:46.487633  3721 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:46.489704  3721 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.491410  3721 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.559270   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1688) took 1153 ms. Trace:
I20260709 13:13:46.532382   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1740) took 1029 ms. Trace:
W20260709 13:13:46.563710  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 192622us
I20260709 13:13:46.573441   999 rpcz_store.cc:276] 0709 13:13:45.406072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.966043 (+559971us) service_pool.cc:224] Handling call
0709 13:13:45.992290 (+ 26247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.992692 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.992729 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.001806 (+  9077us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.408221 (+406415us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.408272 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.408369 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.559056 (+150687us) raft_consensus.cc:1685] finished
0709 13:13:46.559076 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.559210 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2501,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":622,"thread_start_us":334309,"threads_started":1,"wal-append.queue_time_us":466862}
I20260709 13:13:46.575188   925 rpcz_store.cc:276] 0709 13:13:45.503332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.937599 (+434267us) service_pool.cc:224] Handling call
0709 13:13:45.969233 (+ 31634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.096903 (+127670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.096934 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.120903 (+ 23969us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.400576 (+279673us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.400631 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.400732 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.532021 (+131289us) raft_consensus.cc:1685] finished
0709 13:13:46.532055 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.532297 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199815,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":332997,"threads_started":2,"wal-append.queue_time_us":329215}
I20260709 13:13:46.468464  3423 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:46.504408   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 261655us
W20260709 13:13:46.540016   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 299166us
I20260709 13:13:46.580793   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 318) took 1013 ms. Trace:
I20260709 13:13:46.581259   923 rpcz_store.cc:276] 0709 13:13:45.567208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.038742 (+471534us) service_pool.cc:224] Handling call
0709 13:13:46.038958 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.039145 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:46.240018 (+200873us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.580342 (+340324us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.580415 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:46.580481 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.580572 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.580724 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16603,"prepare.queue_time_us":328158,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"thread_start_us":523050,"threads_started":2}
I20260709 13:13:46.584331   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1739) took 1081 ms. Trace:
I20260709 13:13:46.578169   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 319) took 1010 ms. Trace:
I20260709 13:13:46.586699   929 rpcz_store.cc:276] 0709 13:13:45.567557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.071006 (+503449us) service_pool.cc:224] Handling call
0709 13:13:46.071357 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.071624 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:46.240724 (+169100us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.577360 (+336636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.577435 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:46.577489 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.577813 (+   324us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.578098 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13748,"prepare.queue_time_us":308197,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"thread_start_us":491260,"threads_started":2}
I20260709 13:13:46.589407   929 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: [6.5-7.6]
I20260709 13:13:46.562350   927 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:46.607734   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.610651   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.613487   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:46.486732  3727 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:46.584856   930 rpcz_store.cc:276] 0709 13:13:45.503039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.730413 (+227374us) service_pool.cc:224] Handling call
0709 13:13:45.744747 (+ 14334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:45.979210 (+234463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.979249 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:45.981048 (+  1799us) log.cc:844] Serialized 52 byte log entry
0709 13:13:46.259931 (+278883us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:46.564233 (+304302us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.583909 (+ 19676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.584016 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.584076 (+    60us) raft_consensus.cc:1685] finished
0709 13:13:46.584093 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.584256 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79349,"prepare.queue_time_us":127317,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":417128,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"thread_start_us":618960,"threads_started":3,"wal-append.queue_time_us":319397}
I20260709 13:13:46.562803  3214 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:46.577648  3707 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:46.623886  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.602133  3677 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.626590   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.617547  3361 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.521160  3709 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.627996   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.579730  3632 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:13:46.616077  3735 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:46.634729   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:46.626042  3482 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.641057  3709 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.615847  3423 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.642590   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.629774  3731 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:46.652148  3738 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:46.656445   895 leader_election.cc:400] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5.
I20260709 13:13:46.658003  3727 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.659466  3727 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.638263  3361 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.660193  3648 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:46.665146  3648 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.668246  3648 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.669307   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 1814) took 1149 ms. Trace:
I20260709 13:13:46.671428  3744 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:46.673094   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:46.675771  3747 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.676556  3493 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:46.681643  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:46.638846  3632 consensus_queue.cc:1243] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:46.680503   852 rpcz_store.cc:276] 0709 13:13:45.519796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.523031 (+  3235us) service_pool.cc:224] Handling call
0709 13:13:46.669083 (+1146052us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540023a854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:46.669237 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084884224}
I20260709 13:13:46.693331   852 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:46.694926  3423 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.695704  3731 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:46.696988   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1640) took 1277 ms. Trace:
I20260709 13:13:46.697366   922 rpcz_store.cc:276] 0709 13:13:45.419584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.419722 (+   138us) service_pool.cc:224] Handling call
0709 13:13:45.424214 (+  4492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.460677 (+ 36463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.460714 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.461219 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.631791 (+170572us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:45.631834 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.696582 (+1064748us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54001699d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:46.696632 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.696678 (+    46us) raft_consensus.cc:1685] finished
0709 13:13:46.696692 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.696927 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1297,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":73722752,"thread_start_us":170201,"threads_started":1,"wal-append.queue_time_us":293058}
I20260709 13:13:46.700258  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:46.691680  3709 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.628084  3707 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.703258   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.703773   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:46.679939  3750 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:46.705503  3750 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.709352   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:46.713146  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:46.713315   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.714684   852 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:46.678025  3715 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:46.685534  3747 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.704764  3707 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.643558  3482 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.741736  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1684) took 1392 ms. Trace:
I20260709 13:13:46.742192  1000 rpcz_store.cc:276] 0709 13:13:45.348934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.349050 (+   116us) service_pool.cc:224] Handling call
0709 13:13:45.352579 (+  3529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.498956 (+146377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.741494 (+1242538us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001ab3d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:46.741676 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298681088}
I20260709 13:13:46.745434  3747 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.748719  3214 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.751919   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:46.753130   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.755787  3498 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:46.756537   846 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:46.756388  3498 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.757453   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:46.758584  3498 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.759024   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.759722  3632 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:46.711959  3750 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.759830   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.758508   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:46.764712   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:46.767421   846 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:46.771054   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:46.773100   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:46.773955   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:46.773299   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:46.775211   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:46.777813   925 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:46.777863   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:46.778010   929 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:46.776182   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:46.780652   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:46.787730   852 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:46.794265   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:46.796085   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:46.798249   923 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:46.798750   923 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:46.805305  3721 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:46.806429  3715 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:46.806797  3715 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:46.777513  3756 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:46.766971  3652 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.807690  3280 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.792346  3609 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:46.767634  3097 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:46.809592   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:46.815721   925 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:46.815577   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:46.768821  3746 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:46.799710   923 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.814489  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:46.822850   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:46.817204  1000 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:46.817231  3280 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.820292  3691 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:46.821259   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:46.786052  3743 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:46.814509  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 137900us
W20260709 13:13:46.860623   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 78850.3us
I20260709 13:13:46.795560  3648 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.811224  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:46.808673  3638 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:46.822070  3746 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.876816   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1744) took 1372 ms. Trace:
I20260709 13:13:46.884666   969 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7.
I20260709 13:13:46.885488   969 leader_election.cc:403] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:13:46.885152   924 rpcz_store.cc:276] 0709 13:13:45.504465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.002386 (+497921us) service_pool.cc:224] Handling call
0709 13:13:46.005587 (+  3201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.345247 (+339660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.345276 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.354094 (+  8818us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.565680 (+211586us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.565730 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.565837 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.876627 (+310790us) raft_consensus.cc:1685] finished
0709 13:13:46.876645 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.876759 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35423,"raft.queue_time_us":163851,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":57,"thread_start_us":441540,"threads_started":2,"wal-append.queue_time_us":314632}
W20260709 13:13:46.885763   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 97843.4us
I20260709 13:13:46.888012  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:46.888921  1002 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:46.787850  3632 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.826027  3609 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:46.898381   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:46.809444  3750 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:13:46.904687  1601 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:46.861740  3613 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.860015  3691 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:46.911904  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:46.915457   846 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:13:46.907281   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 126689us
I20260709 13:13:46.824520  3652 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.833201  3097 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.903355  3482 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.887681  3747 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
W20260709 13:13:46.944365  3681 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:13:47.003825  3097 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:47.004753   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.940828  3701 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.044s	user 0.002s	sys 0.003s
I20260709 13:13:46.960497   846 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:46.946696  3634 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.980969  3766 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:46.981830  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1573) took 2367 ms. Trace:
I20260709 13:13:46.982669  3746 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.877213  3715 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.983424  3768 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:46.984826  3280 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:46.985682  3613 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:46.883896  3743 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.988348  3652 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:46.991667  3672 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:13:47.053894  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.890400  3638 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:46.992837  3609 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:46.994478  3482 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:46.981472  3747 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:13:46.999689   996 raft_consensus.cc:2410] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:47.000278  3691 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.013249  1001 rpcz_store.cc:276] 0709 13:13:44.613955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.707506 (+ 93551us) service_pool.cc:224] Handling call
0709 13:13:44.707791 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.052256 (+344465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.981586 (+1929330us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54001aaed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:46.981775 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637002496}
W20260709 13:13:47.016278   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 124384us
W20260709 13:13:47.051177   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 174392us
W20260709 13:13:47.012708  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 119226us
I20260709 13:13:47.064397  3777 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:47.064561  2959 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:47.065057  3777 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.067248  3777 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:47.096483   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 189285us
I20260709 13:13:47.098906   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1603) took 1647 ms. Trace:
I20260709 13:13:47.099750   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.055195  3280 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.062923  3613 consensus_queue.cc:1243] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:47.104972   853 rpcz_store.cc:276] 0709 13:13:45.451686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.465690 (+ 14004us) service_pool.cc:224] Handling call
0709 13:13:45.465993 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.466222 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:45.466248 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.466819 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.563709 (+ 96890us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:45.563764 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.563936 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.729982 (+1166046us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:47.073623 (+343641us) raft_consensus.cc:1685] finished
0709 13:13:47.073654 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.098842 (+ 25188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6280960,"thread_start_us":96467,"threads_started":1,"wal-append.queue_time_us":265904}
I20260709 13:13:47.116011  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.116381   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.129840  3743 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.130964  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.131153  3774 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:47.059825  3482 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:47.109542  3607 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:47.110155  3280 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.097580  3604 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.142503   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.104521  3715 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.054431  3768 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.152020   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.133347   846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35b46c75a1e049dd910275709bd62839"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:47.156450   846 raft_consensus.cc:606] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.105011  3097 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:13:47.103708  3760 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:13:47.167279   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 231766us
I20260709 13:13:47.150743  3765 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.171834   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.149127  3652 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:13:47.185418   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1741) took 1681 ms. Trace:
I20260709 13:13:47.186013   926 rpcz_store.cc:276] 0709 13:13:45.503619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.940533 (+436914us) service_pool.cc:224] Handling call
0709 13:13:45.940846 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.204296 (+263450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.185078 (+980782us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54004457d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:47.185336 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591557376}
W20260709 13:13:47.192243  1004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1658) took 4193 ms (client timeout 2206 ms). Trace:
W20260709 13:13:47.192896  1004 rpcz_store.cc:269] 0709 13:13:42.998481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.998611 (+   130us) service_pool.cc:224] Handling call
0709 13:13:47.191912 (+4193301us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b540035b954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:47.192104 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7674979200}
I20260709 13:13:47.194973   998 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:47.196976   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1694) took 1735 ms. Trace:
I20260709 13:13:47.197445   998 rpcz_store.cc:276] 0709 13:13:45.461148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.986578 (+525430us) service_pool.cc:224] Handling call
0709 13:13:47.194471 (+1207893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.195566 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.196693 (+  1127us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540035b950. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:47.196893 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777044864}
W20260709 13:13:47.200258  3705 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 1.238s	user 0.003s	sys 0.001s
I20260709 13:13:47.212282  3537 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:47.212929  3537 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:47.177888  3767 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:47.218451  3423 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.233081  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 133835us
I20260709 13:13:47.231890  3636 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:13:47.227805  3770 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:13:47.218860  3654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.018s	sys 0.039s
I20260709 13:13:47.233036  3767 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:47.239578  3423 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.248961  3776 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.255229  3423 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:47.256886  3702 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.297s	user 0.000s	sys 0.004s
I20260709 13:13:47.257141  3768 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.170123  3482 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.259106  3482 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.259377  3611 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:13:47.245682  3654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.018s	sys 0.039s
I20260709 13:13:47.264967   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.237108  3636 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:47.260332   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1598) took 1855 ms. Trace:
I20260709 13:13:47.249042  3767 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:47.281291   995 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:47.271800   849 rpcz_store.cc:276] 0709 13:13:45.405235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.413628 (+  8393us) service_pool.cc:224] Handling call
0709 13:13:45.414049 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.432961 (+ 18912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.432992 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.433816 (+   824us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.631074 (+197258us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:45.741622 (+110548us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.741966 (+   344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.742050 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:47.256320 (+1514270us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:47.260041 (+  3721us) raft_consensus.cc:1685] finished
0709 13:13:47.260124 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.260258 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1103,"prepare.queue_time_us":93681,"prepare.run_cpu_time_us":4718,"prepare.run_wall_time_us":59890,"raft.queue_time_us":21389,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8137,"spinlock_wait_cycles":23377664,"thread_start_us":196929,"threads_started":1,"wal-append.queue_time_us":306720}
I20260709 13:13:47.267716  3498 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:47.267071  3611 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:47.284796  3636 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.291990  3636 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:47.275525  3638 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.327349  3611 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.328644  3611 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:47.296314  1002 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:13:47.332033  1002 pending_rounds.cc:85] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 7
I20260709 13:13:47.332289  1002 pending_rounds.cc:107] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:13:47.332533  1002 raft_consensus.cc:2889] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:47.319048  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.275935  3652 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:47.320896  3782 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:47.285112  3498 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:47.214118  3537 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.333812  3746 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.326316  3483 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:47.354696  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 516) took 1842 ms. Trace:
I20260709 13:13:47.355219  3766 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:47.337102  3780 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:47.343653  3785 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:13:47.323146  3768 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:47.368777   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.363166  1600 rpcz_store.cc:276] 0709 13:13:45.512607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.513115 (+   508us) service_pool.cc:224] Handling call
0709 13:13:45.513538 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.513949 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.514004 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.515979 (+  1975us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.822320 (+1306341us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b180041fce0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:13:47.089433 (+267113us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:47.089495 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.089612 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:47.354400 (+264788us) raft_consensus.cc:1685] finished
0709 13:13:47.354418 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.354635 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2273,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":2019,"spinlock_wait_cycles":16549120,"thread_start_us":266859,"threads_started":1,"wal-append.queue_time_us":1769391}
W20260709 13:13:47.355769  3784 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:13:47.356114  3750 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:47.360394  3652 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.378048  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.368150  3280 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.382732   921 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:47.373808  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 209769us
W20260709 13:13:47.383908   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.384429   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 216175us
W20260709 13:13:47.386274   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 214543us
I20260709 13:13:47.376947  3750 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:47.378355  3537 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.388157   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.386742   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:47.393006  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 146554us
I20260709 13:13:47.392860  3793 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:47.394887   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.385453  3498 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.381304  3652 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.397943  3652 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:47.399688  3097 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:47.391304  3654 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.400185  3097 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.375281  3483 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.400652  3097 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.402395  3483 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.402882   846 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:47.403415  3781 consensus_queue.cc:1175] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 7 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:47.373958  3780 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.372478  3746 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.380550  3768 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.416733  3097 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.418844  3097 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:47.420373  3776 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.420718   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.421425   854 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:13:47.421572  3750 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.423242  3750 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.380232  3766 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.392875   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.435269   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1799) took 2251 ms. Trace:
I20260709 13:13:47.435848   847 rpcz_store.cc:276] 0709 13:13:45.184067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.184204 (+   137us) service_pool.cc:224] Handling call
0709 13:13:45.196416 (+ 12212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.526556 (+330140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.434937 (+1908381us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54000c99d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:47.435195 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749244288}
W20260709 13:13:47.437960   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.450049  3691 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.437227  3634 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7.
I20260709 13:13:47.453764  3766 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.455571  3625 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.455852  3767 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.458178  3767 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:47.460518  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.464286  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73850.1us
I20260709 13:13:47.465855   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1745) took 1961 ms. Trace:
I20260709 13:13:47.466658   921 rpcz_store.cc:276] 0709 13:13:45.504729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.011511 (+506782us) service_pool.cc:224] Handling call
0709 13:13:46.011911 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.383349 (+1371438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.465534 (+ 82185us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003caad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:47.465768 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3152012544}
W20260709 13:13:47.467972   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.470099  3781 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:13:47.470245   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75435.8us
I20260709 13:13:47.471132   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.472334   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:47.473043   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.474043   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81245.2us
I20260709 13:13:47.486896  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.487295  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.489353   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.486624   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.487759  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.494256   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.495488   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.497571   921 raft_consensus.cc:2410] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:47.499891   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.498131  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.502064  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.502135   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.504858   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.505194   928 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:47.505131   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.506023   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.506206   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.507258   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.507364  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.472620  3791 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.511914  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.524091   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.524930   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:47.514411   922 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:47.525720   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:47.511127  3760 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:47.528422   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.529223   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.529315   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.518440   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.519021   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.474100  3607 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:47.519711  3097 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.520984  3776 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:47.521311  3634 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.516824   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.511475  3767 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:13:47.504251  3786 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:13:47.523346   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.488175  3766 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:47.516824   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:47.495090  3611 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.524263  3716 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:47.532423   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.496595  3781 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.531936   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:47.531152   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.534999   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.536192   894 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.554178   922 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.559839   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.537205   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.523772  3625 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:47.563244   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.564738   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.545629   850 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.498318  3482 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.570724  3482 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.573055  3482 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.497498  3613 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:13:47.550354   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.575299   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.576825   925 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:47.578580  3604 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:13:47.582019  3779 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.333s	user 0.001s	sys 0.003s
I20260709 13:13:47.527834  3483 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:13:47.570106   849 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:47.586083   930 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:13:47.586529   925 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:47.531301  3638 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:47.550639  3097 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.548782  3780 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.561228  3787 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.616528   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.605034   896 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.617879   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.562476  3711 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:47.642231  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.642813   922 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:47.645924  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.646797  1002 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:47.648303  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:47.650002   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.655577   850 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:47.550026  3766 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.604758  3483 consensus_queue.cc:1243] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:47.616341  3677 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.649264  3717 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:47.651422  3427 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.636277  3097 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.677075   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.649128  3787 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:47.688383  3498 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:47.688721  3766 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.690426   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 308) took 4513 ms. Trace:
I20260709 13:13:47.690829   995 rpcz_store.cc:276] 0709 13:13:43.177437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.982920 (+805483us) service_pool.cc:224] Handling call
0709 13:13:45.121461 (+1138541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.282081 (+2160620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.511380 (+229299us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b540022c4d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:47.690218 (+178838us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540022c4d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:47.690367 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7531739904}
I20260709 13:13:47.694013  1596 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:47.668304  3677 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.709795   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.711751  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:47.713917  1002 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.709892  3802 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:47.714932  3802 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:47.728988  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 190954us
I20260709 13:13:47.774961  3710 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:47.752084  1002 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:47.750257  3677 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.731181  3803 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:47.860718   928 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:47.861497  1003 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:47.862013   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.862670  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.863909  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.864616  1002 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:47.864657   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:47.865629  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.866725  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:47.869398  3731 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:47.840927  3718 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:47.872102  3718 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.874296  3718 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.879168   923 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:47.789095  3746 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.803498  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:47.884502  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 81082.3us
I20260709 13:13:47.863493  1003 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:47.757200   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:47.805311  3750 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:47.806542   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:47.890583   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.807122   925 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:47.808681  1001 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:47.891029   929 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:13:47.739534  3800 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:47.724206  3802 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.757055   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.707728  3776 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:47.765149  3768 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:47.745038  3787 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:47.946643   923 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:13:47.739675   996 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:47.948961   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:47.950057   996 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:47.951027   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.740537  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.808825   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.770632  3632 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:47.810813   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:47.735379   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59494.5us
I20260709 13:13:47.745611  3097 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:47.841792   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:47.853099  1596 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:47.854866   997 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:47.855868  1000 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:47.859009   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:47.863981  3625 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.880298  3607 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:47.766018  3611 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:47.804571  3775 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:47.886840  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.792714  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:47.882474   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 69080.5us
I20260709 13:13:47.927170  3731 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.921808  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.928664  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.940665   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.941126  1597 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:13:47.848486  3710 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:47.946156  3776 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.964495  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:47.954034   996 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:47.954351   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:48.049023   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 94757.4us
W20260709 13:13:48.052031  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.997743  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:48.058808  1596 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:47.995383  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:48.060352  1000 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:48.061432  1597 pending_rounds.cc:85] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 5
I20260709 13:13:48.061756  1597 pending_rounds.cc:107] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:13:48.062053  1597 raft_consensus.cc:2889] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:48.019289   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.995518  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.066265  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:48.068166  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.954707   923 pending_rounds.cc:85] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 3
I20260709 13:13:47.957979   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:47.921267  3604 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:47.921626  3721 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.890795  3750 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:47.961248   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:47.961933   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.905896   999 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:48.043696  3775 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.044648   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:47.733472  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:48.071885   923 pending_rounds.cc:107] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:13:48.079066   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.093259   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.922689  3743 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:13:47.929903  3677 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:13:48.095821  1001 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:13:48.111416   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.163513   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 205584us
I20260709 13:13:48.165493   969 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:48.155375   923 raft_consensus.cc:2889] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:48.167313   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1668) took 1369 ms. Trace:
I20260709 13:13:48.167773   923 rpcz_store.cc:276] 0709 13:13:46.797417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.797715 (+   298us) service_pool.cc:224] Handling call
0709 13:13:46.798088 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.878776 (+1080688us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b58004c7cd0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02a9b0aa 00007f0d02b1d447 00007f0d02b35fe7 00007f0d02b377de 00007f0d02b41b49 00007f0d02b42416 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4
0709 13:13:48.166996 (+288220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.167248 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103752448}
I20260709 13:13:48.170699   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:48.172698  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 177307us
I20260709 13:13:47.960078  3607 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:48.189523  3493 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:48.189966  3493 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:48.189842   750 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 5, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.191375  3709 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:47.995724  3611 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.191433   750 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.192044  3709 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.192812   750 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:48.193424  3611 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:48.194074  3709 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:48.194137   750 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:48.155699   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 62583.3us
I20260709 13:13:48.067047  1003 raft_consensus.cc:2376] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6. Re-sending same reply.
I20260709 13:13:48.114357  1002 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:48.158241   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 414107us
I20260709 13:13:48.190181  3607 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:47.944244  3719 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.187s
I20260709 13:13:48.217026  1002 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:48.156375   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:48.204747   854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.060s	sys 0.079s
I20260709 13:13:48.208773  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.218164   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.218403  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
W20260709 13:13:48.221814   854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.061s	sys 0.079s
I20260709 13:13:48.220813   997 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:48.224797  1001 raft_consensus.cc:606] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.225354  1001 raft_consensus.cc:614] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:48.119378  3427 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:48.223891  1004 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.216801  1000 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:48.207815   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 4 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.096856  3604 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.221063  1002 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:13:48.219503   821 leader_election.cc:336] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.563s (SENT)
I20260709 13:13:48.238211   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:47.960314  3715 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.263s
W20260709 13:13:48.280733   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 47122.7us
I20260709 13:13:48.161513  3780 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.235217   996 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:48.216271  3730 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:13:48.236431   750 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 4, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 4 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:13:48.305480  1601 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.006s	sys 0.028s
W20260709 13:13:48.305824  1601 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.006s	sys 0.028s
I20260709 13:13:48.306026   750 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:48.239048  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.307873  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.309985  1002 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:48.217644  3767 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.223322  3709 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:48.312172   750 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:48.226950  3781 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:48.159153  3721 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.202963  3636 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 13:13:48.323868   750 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 7, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:48.328492  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:48.409695  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 81290.4us
I20260709 13:13:48.410421  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.279028  3604 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:48.329818  3780 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:48.401255  3769 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:13:48.113859  3731 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.302743  2959 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:48.385316  3806 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:13:48.318739  3427 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.226536  3483 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:13:48.407284  1004 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:13:48.333786  3766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.051s
I20260709 13:13:48.377110  3097 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:48.331311  3672 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.405715   851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.016s
W20260709 13:13:48.449380   851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.008s	sys 0.016s
I20260709 13:13:48.413520   750 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.415271   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.394532  3677 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:48.326833  3715 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:13:48.293201  3607 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.306499  3750 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.334998  3709 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.430619   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:48.417558  3781 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:48.490015   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 59469.2us
I20260709 13:13:48.419888  3280 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.421324  3797 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.236488  3710 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.421911  3805 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.454869   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.457346   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:48.435377   738 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 597687e464174d1ea6043161e037d240 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:13:48.444134  3766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.004s	sys 0.051s
I20260709 13:13:48.451715   750 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:48.470350   998 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.013s	sys 0.021s
W20260709 13:13:48.521435   998 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.013s	sys 0.021s
I20260709 13:13:48.453212  3097 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.455293  3672 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.459085  3498 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.455440  3743 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.452690  3677 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:48.452973  3776 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
W20260709 13:13:48.488605   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 269110us
W20260709 13:13:48.553596   821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.619s (SENT) [suppressed 29 similar messages]
I20260709 13:13:48.517347  3775 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:48.513118   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.443527  3767 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:13:48.473346  3636 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:48.462939  3632 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:13:48.522091   750 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:48.525032  3709 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:48.534534   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 316496us
I20260709 13:13:48.575062   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.576762   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:48.540457   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 320996us
W20260709 13:13:48.579821   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.496543  3802 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:48.582021   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:48.555503   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.446513  3427 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.534314  3710 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.599735   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:48.542472  3750 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:48.539810   738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 106010us
I20260709 13:13:48.499704  3803 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:13:48.607079   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:48.568804   750 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.610008   750 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.584606   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:48.595877   853 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:48.612814   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.575305  3775 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:48.594146  3818 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:13:48.614382   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.597473   928 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:13:48.615192   928 pending_rounds.cc:85] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:13:48.615507   928 pending_rounds.cc:107] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:13:48.615810   928 raft_consensus.cc:2889] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:48.566831   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.574276  3707 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.563773  3711 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.594811  3802 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:48.627068  3820 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 13:13:48.635834  3821 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:13:48.553957  3743 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:48.665599   821 leader_election.cc:336] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 2.206s (SENT)
I20260709 13:13:48.594991  3768 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.644251  3731 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:48.604699  3781 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.595333  3766 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.637105  3672 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.595693  3780 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:48.676764   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.695008   929 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:48.700939   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.725417   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.726547   853 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:48.727105   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 226067us
I20260709 13:13:48.665912  3709 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.706830   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 108032us
W20260709 13:13:48.709645   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44081.2us
I20260709 13:13:48.581239  3636 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.668901   750 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 6, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:48.716611   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 259345us
I20260709 13:13:48.737510   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:48.737847   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.736878   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.673286  3722 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:48.696801  3825 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.696048  3802 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:48.741236   855 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:13:48.701066  3831 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:48.755089   922 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:48.738170   750 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.757085   926 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:48.762176   750 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:48.779141   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1596) took 1015 ms. Trace:
I20260709 13:13:48.696501  3707 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.732105  3731 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:48.615574  3677 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.748687  3830 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.787577   750 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:13:48.702476  3711 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.780599   998 rpcz_store.cc:276] 0709 13:13:47.807707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.808556 (+   849us) service_pool.cc:224] Handling call
0709 13:13:47.808808 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.809018 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:47.809307 (+   289us) log.cc:844] Serialized 14 byte log entry
0709 13:13:48.522036 (+712729us) log.cc:844] Serialized 14 byte log entry
0709 13:13:48.778767 (+256731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:48.778830 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:48.778876 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.778953 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.779085 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21047,"prepare.queue_time_us":31278,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":37508,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":825,"spinlock_wait_cycles":23032960,"thread_start_us":968752,"threads_started":2,"wal-append.queue_time_us":784502}
I20260709 13:13:48.753407  3766 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.796733   750 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 2, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:48.805505   927 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:48.836248   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.840409   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.778779  3638 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:48.798950   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1612) took 2012 ms. Trace:
I20260709 13:13:48.732679  3780 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.739523  3833 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
W20260709 13:13:48.867245   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 114094us
I20260709 13:13:48.751104  3709 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.759895  3722 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.797406  3423 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:48.778576  3768 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:13:48.799306  3483 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:48.762751  3825 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.866717  3677 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.971737  3780 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.776777  3831 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.795872  3731 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.892094  3785 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.898044  3097 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.934448   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:48.932524   848 rpcz_store.cc:276] 0709 13:13:46.787032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.788086 (+  1054us) service_pool.cc:224] Handling call
0709 13:13:46.788402 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:47.469956 (+681554us) spinlock_profiling.cc:243] Waited 9.5 ms on lock 0x7b18000f3040. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0d026e74bc 00007f0cffa3548b 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa4f455 00007f0d02b37060 00007f0d02b37bfb 00007f0d02b4264b
0709 13:13:47.470081 (+   125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.470106 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:47.772787 (+302681us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c00000c88. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:13:47.773275 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.220871 (+447596us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:48.220908 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.220979 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.798736 (+577757us) raft_consensus.cc:1685] finished
0709 13:13:48.798758 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.798890 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5437,"raft.queue_time_us":102304,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7498,"spinlock_wait_cycles":45040640,"thread_start_us":558505,"threads_started":2,"wal-append.queue_time_us":675064}
I20260709 13:13:48.946133   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:48.802979  3830 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.985879  3830 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.986660  3830 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.956537   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.991565  3835 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:48.969843  3837 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:13:48.970397  3838 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:13:48.970679  3483 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.993631   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1715) took 1123 ms. Trace:
I20260709 13:13:48.993486  3830 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:48.994239  3831 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.994951  3483 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:48.933111  3427 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.995672  3427 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.995975  3830 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.996352  3731 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:48.996133  3427 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:48.992690   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46651.6us
I20260709 13:13:48.997962   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:48.999575   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:49.000317   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 75848.7us
W20260709 13:13:48.967154  3609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.631s	user 0.032s	sys 0.035s
I20260709 13:13:48.994122   997 rpcz_store.cc:276] 0709 13:13:47.869946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.228800 (+358854us) service_pool.cc:224] Handling call
0709 13:13:48.229165 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.304592 (+ 75427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.304622 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.305058 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.701498 (+396440us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:48.701539 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.701608 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.993395 (+291787us) raft_consensus.cc:1685] finished
0709 13:13:48.993415 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.993571 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117141,"raft.queue_time_us":75189,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":14976,"thread_start_us":305415,"threads_started":1,"wal-append.queue_time_us":413952}
I20260709 13:13:49.008699   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.013792   998 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:48.953068  3638 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:48.968303  3423 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.053947  3846 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.054731  3840 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.056282  3482 raft_consensus.cc:993] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:13:49.057222  3709 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.010488  3483 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.010799  3831 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:49.060094  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.060956  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.060110  3429 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.081748  3429 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.083953  3429 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.084417  3429 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.084842  3429 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.063773  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.065835  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:49.067065  3797 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:13:49.086308  3797 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:49.068003  3839 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:13:49.068681  3427 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.052625  3707 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:49.069998   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 61378.4us
I20260709 13:13:49.087978  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1721) took 1216 ms. Trace:
I20260709 13:13:49.088814   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:49.010589  3743 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:49.070622   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.023892   997 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:13:49.090023   997 pending_rounds.cc:85] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 3
I20260709 13:13:49.090252   997 pending_rounds.cc:107] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 13:13:49.090297   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.090548   997 raft_consensus.cc:2889] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:49.071496   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1722) took 1150 ms. Trace:
I20260709 13:13:48.972806  3850 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:13:49.007771  3731 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.071704  3648 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.992740  3722 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.008026  3785 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:49.021150  3609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.686s	user 0.033s	sys 0.035s
I20260709 13:13:49.091384   998 rpcz_store.cc:276] 0709 13:13:47.920711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.013033 (+1092322us) service_pool.cc:224] Handling call
0709 13:13:49.013473 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.071166 (+ 57693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.071425 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:49.107236   998 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:49.074337  3607 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 13:13:49.088315  1002 rpcz_store.cc:276] 0709 13:13:47.884730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.315470 (+430740us) service_pool.cc:224] Handling call
0709 13:13:48.315762 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.403350 (+ 87588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.403487 (+   137us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.423728 (+ 20241us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.600677 (+176949us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:48.600733 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.600824 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.087752 (+486928us) raft_consensus.cc:1685] finished
0709 13:13:49.087772 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.087919 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69761,"raft.queue_time_us":85631,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":13036,"thread_start_us":153276,"threads_started":1,"wal-append.queue_time_us":459104}
I20260709 13:13:49.099745   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.098119  3482 raft_consensus.cc:1081] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:13:49.104537  3743 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:13:49.116900  3825 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.102451  3848 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.123896   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.131738  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.132819  1596 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:49.098634  3707 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:49.099789  3711 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.134845  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.136575  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.140291  1596 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:49.140365  1592 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:13:49.138235   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.138237  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.138806  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.073011  3846 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.144238  1595 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:49.131487  3483 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:49.095643  3731 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.154281   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.157123  3429 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.153342  3848 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:49.158581  3822 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.688s	user 0.001s	sys 0.008s
I20260709 13:13:49.158003  3848 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.165957  3848 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.150215   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:49.167568  1004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1700) took 1435 ms (client timeout 1763 ms). Trace:
W20260709 13:13:49.168083  1004 rpcz_store.cc:269] 0709 13:13:47.732156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.733183 (+  1027us) service_pool.cc:224] Handling call
0709 13:13:49.167306 (+1434123us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b5400359b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:49.167501 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152778624}
I20260709 13:13:49.170517  1001 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:49.171641  3841 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:13:49.077576  3840 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.171509   848 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:13:49.172168  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1607) took 1278 ms. Trace:
I20260709 13:13:49.172573  1001 rpcz_store.cc:276] 0709 13:13:47.893657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.318386 (+424729us) service_pool.cc:224] Handling call
0709 13:13:49.170301 (+851915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.170978 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.171988 (+  1010us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400359b50. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:49.172114 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958544640}
I20260709 13:13:49.173714  3840 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.174813  3493 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.176004  3493 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:49.176729  3638 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.201086   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.201086  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:49.202922   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57008.6us
I20260709 13:13:49.140550  3722 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:49.183928   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.208542   851 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:49.177834   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.202328   847 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:49.210258   848 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:49.142546  3831 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.144362  3856 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:49.145495  3711 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:13:49.144452  3780 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.099s
W20260709 13:13:49.265633   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 110451us
I20260709 13:13:49.214478  3722 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:49.257958  3604 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.824s	user 0.009s	sys 0.004s
I20260709 13:13:49.266412   894 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:49.286859   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:49.288131   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:49.289269   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:49.141610  3707 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.306401  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.307394  1001 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:49.291214  3604 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:49.301755  3860 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:13:49.295423   850 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:13:49.302731  3815 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.768s	user 0.002s	sys 0.000s
I20260709 13:13:49.299764   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1717) took 1429 ms. Trace:
I20260709 13:13:49.299080   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1713) took 1429 ms. Trace:
I20260709 13:13:49.315631   999 rpcz_store.cc:276] 0709 13:13:47.869371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.156033 (+286662us) service_pool.cc:224] Handling call
0709 13:13:49.298850 (+1142817us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003e3354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:49.299013 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2319596928}
W20260709 13:13:49.312695   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 584707us
I20260709 13:13:49.315482   996 rpcz_store.cc:276] 0709 13:13:47.870532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.295349 (+424817us) service_pool.cc:224] Handling call
0709 13:13:48.332412 (+ 37063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.332598 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:48.395522 (+ 62924us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.299343 (+903821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.299424 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:49.299475 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.299571 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.299711 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":703979,"prepare.queue_time_us":62788,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":710036,"thread_start_us":199430,"threads_started":1,"wal-append.queue_time_us":402946}
I20260709 13:13:49.300900   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.319731   894 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:49.319706   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 581871us
I20260709 13:13:49.321110   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.321630  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.321908  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:49.324163   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 96539us
I20260709 13:13:49.324523  1595 raft_consensus.cc:2410] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:49.325452   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:49.292948  3825 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:49.295893  3717 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:49.234587  3856 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.320932  3672 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.311849  3775 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:49.328953  1601 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:49.330303   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:49.310806  3709 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:49.328558   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.324136  3423 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:49.340683  3859 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:13:49.330124  3717 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.341804  3717 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:49.344503   929 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:49.345443  3834 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:49.345841  3834 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.346185  3834 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:49.349403  3856 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.331802  3677 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:49.350891  3677 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.339334   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1829) took 1831 ms. Trace:
I20260709 13:13:49.361971  3858 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:49.362210  3778 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.362536  3858 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.363025  3858 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:49.362834  3778 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:49.362556  3812 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.918s	user 0.001s	sys 0.000s
W20260709 13:13:49.364485   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.365172  3845 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.390s	user 0.001s	sys 0.000s
I20260709 13:13:49.365250  3604 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.366037  3865 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:49.352249   854 rpcz_store.cc:276] 0709 13:13:47.507549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.507686 (+   137us) service_pool.cc:224] Handling call
0709 13:13:47.507991 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:47.584028 (+ 76037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.584053 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:47.584494 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.225066 (+640572us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:48.225835 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.170140 (+944305us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54002b8c54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:49.170191 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.302068 (+131877us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:49.339080 (+ 37012us) raft_consensus.cc:1685] finished
0709 13:13:49.339109 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.339264 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17075,"raft.queue_time_us":1882,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6311,"spinlock_wait_cycles":189591552,"thread_start_us":640112,"threads_started":1,"wal-append.queue_time_us":881624}
I20260709 13:13:49.334179  3711 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:49.351257  3677 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:49.312678  3716 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:49.338446  3672 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.337509  3709 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:49.358506  3750 consensus_meta.cc:287] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.000s
I20260709 13:13:49.384557  3604 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:49.386776   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 65733.6us
I20260709 13:13:49.394578   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.351315  3707 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.405449   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.376416  3423 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.338531  3775 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.366631  3865 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:49.605625   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.623179   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.390589  3709 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:49.407223   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.553943   996 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:13:49.391201  3731 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:49.677162   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.525454   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 133286us
W20260709 13:13:49.533581   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 139081us
W20260709 13:13:49.715699   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 410801us
W20260709 13:13:49.613277  3869 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:13:49.725525  3865 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.405807  3604 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.728224  3775 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:49.728451  3872 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:13:49.688562  3423 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.729928  3604 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.731088  3609 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:49.733774  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:49.643383  3874 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:13:49.548465  3611 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.504110  3707 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.744360  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:49.504195  3632 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.760398  3842 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.760680  3632 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.763029  3632 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.751288  3709 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.592419  3672 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.477156  3648 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:49.764539  3786 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:49.765102  3648 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.765457   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.766003  3672 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.766448   846 raft_consensus.cc:2410] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:49.766526  3672 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.767427  3648 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.768219  3707 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.769209  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:49.769927  3672 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:49.770478  3707 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.759843  3609 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.757858  3803 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:49.788592  3746 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:49.789067  3746 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:49.789199  3721 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:49.789896  3493 consensus_meta.cc:287] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.004s
I20260709 13:13:49.790110  3648 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:49.773099   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.790639  1600 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:49.791152   846 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:49.430280  3654 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.775648  3709 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.778685  3609 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.802286  3609 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:49.781134  3875 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:13:49.785284  3802 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.012s	user 0.011s	sys 0.000s
I20260709 13:13:49.553319  3785 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.793123   969 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:49.807511   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1728) took 1487 ms. Trace:
I20260709 13:13:49.808007   997 rpcz_store.cc:276] 0709 13:13:48.319864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.091374 (+771510us) service_pool.cc:224] Handling call
0709 13:13:49.091710 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.359518 (+267808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.359549 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.373988 (+ 14439us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.575403 (+201415us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:49.575461 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.575621 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.780481 (+204860us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:49.804446 (+ 23965us) raft_consensus.cc:1685] finished
0709 13:13:49.804476 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.807453 (+  2977us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":407830,"raft.queue_time_us":275427,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":13244,"spinlock_wait_cycles":56337152,"thread_start_us":190187,"threads_started":1,"wal-append.queue_time_us":249424}
I20260709 13:13:49.720763  3498 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.794706   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1682) took 1213 ms. Trace:
I20260709 13:13:49.796919  3834 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.799763  3423 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.507647  3781 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.487814  3767 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.549015  3846 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.790659  3648 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.811045   921 rpcz_store.cc:276] 0709 13:13:48.580896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.582296 (+  1400us) service_pool.cc:224] Handling call
0709 13:13:48.582651 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.775484 (+192833us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:48.909301 (+133817us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.256171 (+346870us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.256244 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:49.256303 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.794504 (+538201us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:49.794545 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.794653 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27051,"prepare.queue_time_us":197650,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":17394,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2876,"spinlock_wait_cycles":442751872,"thread_start_us":439093,"threads_started":2,"wal-append.queue_time_us":380394}
I20260709 13:13:49.793725  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:49.804083  3802 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.475474  3717 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.829012  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 332) took 1403 ms. Trace:
I20260709 13:13:49.829412  1002 rpcz_store.cc:276] 0709 13:13:48.425439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.131179 (+705740us) service_pool.cc:224] Handling call
0709 13:13:49.131520 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.360027 (+228507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.360056 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.360482 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.399220 (+ 38738us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:49.399275 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.399356 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.828778 (+429422us) raft_consensus.cc:1685] finished
0709 13:13:49.828799 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.828949 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":355397,"raft.queue_time_us":235633,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":12234,"thread_start_us":38440,"threads_started":1,"wal-append.queue_time_us":154120}
I20260709 13:13:49.583055  3864 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.822449   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:49.834754   849 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:49.817677  3781 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.816147  3632 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:49.816809  3498 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.789953  3746 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.845525  3632 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.828817  3767 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:49.861702   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 109457us
I20260709 13:13:49.863226   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.850580  3677 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.859701  3632 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.844650  3864 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.895673   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.896886   925 raft_consensus.cc:2368] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8.
I20260709 13:13:49.887451  3746 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:49.899091  3880 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:49.901302   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:49.886384  3772 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:49.888643  3654 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:49.906659  3880 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:13:49.909403  3852 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.909972  3852 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:49.916960  3858 consensus_meta.cc:287] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.003s
I20260709 13:13:49.923837   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.924546   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.929739   926 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:49.929622   921 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:49.928043   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.932222   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.933009   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.933478   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.937180   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:49.937916   921 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:49.939148   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:49.933840   930 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:49.939853   928 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:49.941915   927 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:49.918418  3880 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.946259   921 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:13:49.919337  3881 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:49.912647  3772 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:49.954658  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:49.955686   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.956773  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.957986   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:49.958956   997 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:49.959878   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:49.960752   995 raft_consensus.cc:2410] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:49.961431  1003 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:49.955715   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.941844  3834 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.967888  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:49.972690  1002 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:49.950752  3852 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:49.931629  3858 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.952116  3802 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.947629  3880 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.961000  3765 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:49.950138  3876 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:49.952270  3717 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.973886  3498 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:49.975279  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:49.989128  3880 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:49.971012  3781 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:49.970885  3776 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:49.971192  3648 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:49.994462  1002 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:49.992420   929 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:13:49.998420  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:49.999133  1002 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:50.000402  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.001420  3864 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.016024  3718 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:50.016609  3718 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.017395  3852 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:50.018731  3718 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.021508  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.022172  1003 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:50.024214   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.024950   995 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:50.025767   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.026291   995 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:50.027089   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.028474   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.028798  3858 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.033563   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.034492   999 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:50.035743  3767 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.036226  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 525) took 1708 ms. Trace:
I20260709 13:13:50.036736  1601 rpcz_store.cc:276] 0709 13:13:48.327446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.362687 (+ 35241us) service_pool.cc:224] Handling call
0709 13:13:48.425546 (+ 62859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.336232 (+910686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.035993 (+699761us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400591d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:50.036171 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077097216}
I20260709 13:13:50.040575  3772 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.041532  3778 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.045892   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:50.047953  3672 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.048324  3672 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.048666  3672 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:49.992916  3876 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:49.995146  3776 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.057548  3776 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.036302  3677 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.060997  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.061945  1003 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:50.063071  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.071272  1003 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:49.991161  3765 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:50.064630  3829 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.431s	user 0.003s	sys 0.003s
I20260709 13:13:50.077875   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.092658   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.094051   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.092305  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.096942  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.099290  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.100244  1003 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.102097   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.083051  3537 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.109179   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.096506  3858 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.113085   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.095214  3654 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.110219   995 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:50.115692  3858 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.116590  3864 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:50.104873  3775 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.117178  3864 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:50.117622  3775 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.118270  3864 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.114974  3427 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:13:50.112895  3537 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.119803  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.106963  3718 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:50.108567  3711 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:50.126838  3717 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:50.126919  3765 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.127384  3717 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:50.128629  3876 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.132671  3632 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.133558  3636 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.134296  1002 raft_consensus.cc:2368] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:13:50.134719  3632 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.124488  3718 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:50.136061  3718 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.123589  3654 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.136441  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.137025  1002 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:13:50.138751  2959 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:50.139113  2959 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:50.139271  3537 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.128289  3717 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.139943  2959 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.140508  3717 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.140853  2959 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:50.123024  3858 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.147821  3677 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:50.149739  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.150529  1004 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:50.151281  3722 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.151938  3775 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:50.152851   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.153616   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.154050   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.154656   925 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:13:50.145416  3813 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.754s	user 0.003s	sys 0.023s
I20260709 13:13:50.154894   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.158007   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.159389   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.159941   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.161412   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.168547   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.170400   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:50.170539   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.171082   927 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:50.162220   924 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:50.160094   928 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:50.180428   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.131489  3881 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:50.179957  3858 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.197319  3654 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.198578   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:50.200413  3817 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.807s	user 0.000s	sys 0.006s
I20260709 13:13:50.179595   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:50.188968   998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.014s	sys 0.037s
W20260709 13:13:50.202343   998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.015s	sys 0.038s
I20260709 13:13:50.202945   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.203713   999 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:50.204671   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.205549   999 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:50.205439   927 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:50.207767   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.210366   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.211347   927 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:50.207885   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.214274   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.215528   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.215909   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.216622   853 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:50.215399   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.217408   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.217957   999 raft_consensus.cc:2376] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7. Re-sending same reply.
I20260709 13:13:50.218731   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.218766   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.219347   999 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:50.220046   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.220494   999 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:50.221082   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.221859   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1836) took 2558 ms. Trace:
I20260709 13:13:50.222276   852 rpcz_store.cc:276] 0709 13:13:47.663840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.663980 (+   140us) service_pool.cc:224] Handling call
0709 13:13:47.664299 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:47.667208 (+  2909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.667243 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:47.669240 (+  1997us) log.cc:844] Serialized 26 byte log entry
0709 13:13:47.965281 (+296041us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:47.965352 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.965447 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.144863 (+2179416us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:50.216104 (+ 71241us) raft_consensus.cc:1685] finished
0709 13:13:50.216129 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.221791 (+  5662us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2569,"raft.queue_time_us":3086,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3287,"spinlock_wait_cycles":9566464,"thread_start_us":282250,"threads_started":1,"wal-append.queue_time_us":602954}
I20260709 13:13:50.225535  3775 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.229432   928 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:50.231158   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.208321  3677 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:50.216087  3765 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.241488  2959 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.242117  2959 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:50.251993   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.258657  2959 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.222584  3881 raft_consensus.cc:2764] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:50.239176   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.236735   926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35b46c75a1e049dd910275709bd62839"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42390
I20260709 13:13:50.265113   926 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:50.265573   926 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.239195  3677 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.265831   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.223850  3672 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.256254  3746 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.280323  3677 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.282902   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.286198   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.260603  3766 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:50.270465  3778 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.303754   928 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:50.260025  3781 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:13:50.283574  3881 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.312894  3778 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.312646  3766 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.237533  3632 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:50.307366   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.301975  1003 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:50.318675  3778 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.311946  3882 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:50.326097  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.302551   926 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.333988   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.319489  3881 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.306653  3746 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.337777   996 raft_consensus.cc:606] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.338537   996 raft_consensus.cc:614] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:50.339380   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.339882   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.341954   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.342720   925 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:50.340004   930 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:50.345999   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.325773  3876 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:50.355275   926 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.358574   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.351542  3654 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:50.362985  3654 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:50.363796  3766 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.352089  3881 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.364660  3832 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.364213  3654 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.366089  3654 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.337332  3771 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:50.368094  3771 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:50.369338  3855 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:50.377193   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.378209   846 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:50.380029   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.380839   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:50.381412   852 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:50.382304   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.383935   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.383926  3842 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:50.384621  3842 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.384927  3842 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.385866  3746 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.385993   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.386670   854 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:50.379302   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.388907   846 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:50.389031   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.391547   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.332850  3882 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:50.397087   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.398031   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.403085  3803 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.404635   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.405591   850 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:50.355909  3876 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:50.419337   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.419896  3775 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:50.420987  1003 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:50.405954   969 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.395777  3882 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:50.403573  3718 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.369266  3771 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:50.435236  3836 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 1.452s	user 0.000s	sys 0.015s
I20260709 13:13:50.435120  3634 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:50.452021  3718 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.462832  3842 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.464861  3842 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.468793  3882 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.470466  3882 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.470764  3881 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.473419  3881 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:50.509814   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.514237  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1716) took 2643 ms. Trace:
I20260709 13:13:50.515105  1000 rpcz_store.cc:276] 0709 13:13:47.870221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.235563 (+365342us) service_pool.cc:224] Handling call
0709 13:13:48.235864 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.304841 (+ 68977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.304873 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.318382 (+ 13509us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.556158 (+237776us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:48.577127 (+ 20969us) log.cc:844] Serialized 14 byte log entry
0709 13:13:48.596734 (+ 19607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.596840 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.166088 (+569248us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b4800210cd8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:50.313317 (+1147229us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:50.513441 (+200124us) raft_consensus.cc:1685] finished
0709 13:13:50.513489 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.513836 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":271804,"prepare.queue_time_us":37231,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":785592,"raft.run_cpu_time_us":6646,"raft.run_wall_time_us":854737,"spinlock_wait_cycles":52613760,"thread_start_us":157873,"threads_started":1,"wal-append.queue_time_us":561523}
I20260709 13:13:50.519758  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.522167  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.524551   924 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:50.519107   851 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:50.507400   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:50.557451   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50590.1us
I20260709 13:13:50.559593   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.566072  3852 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:50.566342  2959 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:50.568521  2959 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.569314  3746 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:50.566666  3766 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.577144  3766 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.577673  3766 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.581403  3852 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.598083  3672 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:50.588351   852 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:50.604079  3689 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:50.605319  3689 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.608348  3796 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.612430  3796 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.587414  3746 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.600477  3852 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.646739  2959 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.637796  3746 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:50.659302   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.668581   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.670656  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 542) took 1212 ms. Trace:
I20260709 13:13:50.667383  3796 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.671705  1595 rpcz_store.cc:276] 0709 13:13:49.397212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.397434 (+   222us) service_pool.cc:224] Handling call
0709 13:13:49.397754 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.398017 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:49.476695 (+ 78678us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.489059 (+ 12364us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.568714 (+1079655us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b180040f180. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cff9ccc0a 00007f0d0910c219 00007f0cffa54a40 00007f0cffa4f4e4 00007f0cffa4f455 00007f0d02b37060 00007f0d02b49716 00007f0d02b5a287 00007f0d02b5a208
0709 13:13:50.569141 (+   427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:50.569250 (+   109us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:50.569441 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.571409 (+  1968us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.609427 (+ 38018us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1079488,"prepare.queue_time_us":143521,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":1128012,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":1155,"spinlock_wait_cycles":37981440}
I20260709 13:13:50.685021   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.686694  3537 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:50.687168  3537 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.687529  3537 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:50.687775   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.690433   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.691991  3718 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:50.692499  3718 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:50.692649  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.692915  3718 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:50.693738  1004 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.694079   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.697264  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1611) took 1296 ms. Trace:
I20260709 13:13:50.697777  1001 rpcz_store.cc:276] 0709 13:13:49.400728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.400835 (+   107us) service_pool.cc:224] Handling call
0709 13:13:49.401195 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.723863 (+322668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.723894 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.723936 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:49.723977 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.696722 (+972745us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54000eb354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:50.696798 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.697040 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741562624}
I20260709 13:13:50.703104   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:50.703843  3912 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.704175   997 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.704380  3834 raft_consensus.cc:2764] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:50.704790  3912 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:50.714031  3902 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.297s	user 0.008s	sys 0.010s
I20260709 13:13:50.716156  3876 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.718325  3876 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:50.721810   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.736214  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.745744   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:50.736929   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 50812.7us
I20260709 13:13:50.748050   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:50.749292   969 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:50.754513   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.757680  3910 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.118s	user 0.005s	sys 0.000s
I20260709 13:13:50.757438   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.757972  3912 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.759202  1002 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:50.761688  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.764305  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.765892  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.767413  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.768301  3802 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.769069  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.771070  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.771245   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.772823   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:50.779857   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:50.781579   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:50.783433   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.798087  1004 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:50.800064  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.800851  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:50.801268  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:50.797804   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 76208.6us
I20260709 13:13:50.790546  3802 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.807485  1599 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.808328  3857 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:50.808821  3857 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:50.809212  3857 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:50.809576  3834 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.810063  3834 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:50.810437  3834 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.819653  1599 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:50.822589  1004 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:50.824255  1004 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:50.807497  3802 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.836977  1596 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.840798   997 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:50.852708  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:50.853334  1002 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
W20260709 13:13:50.864037   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80743.2us
I20260709 13:13:50.864622   819 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.865052   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:50.865872   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.871536   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 114373us
W20260709 13:13:50.863354  3778 consensus_meta.cc:287] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.004s
I20260709 13:13:50.873374  2959 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:50.840473  3766 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.883666  3844 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:50.884543  3771 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.886365   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1623) took 2299 ms. Trace:
I20260709 13:13:50.886914   853 rpcz_store.cc:276] 0709 13:13:48.587191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.741307 (+154116us) service_pool.cc:224] Handling call
0709 13:13:50.216340 (+1475033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.217052 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.885883 (+668831us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54003b10d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:50.886151 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3391577984}
I20260709 13:13:50.891628  3498 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:50.892465  3498 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:50.893901  3765 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.637s	user 0.003s	sys 0.016s
I20260709 13:13:50.894481  3765 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.895066  3498 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:50.900010  1592 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.900802   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:50.901134   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 172917us
I20260709 13:13:50.863265  3718 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.910249  3722 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:50.910576  3689 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.911018  3722 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:50.913303  3864 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:50.907164   995 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:50.914043  3864 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.914369  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.914521  3864 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:50.914819   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.915238  1600 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:50.915766   995 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:50.916728  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.918848   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.913384  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.915385   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34972 (request call id 12) took 1512 ms. Trace:
I20260709 13:13:50.919800   996 rpcz_store.cc:276] 0709 13:13:49.402364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.534164 (+131800us) service_pool.cc:224] Handling call
0709 13:13:50.914978 (+1380814us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540035a054. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b39853 00007f0d07b0760a 00007f0d0108d7d4 00007f0d0108d762 00007f0d0108d691 00007f0d0108d60c 00007f0d0108c8c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:50.915174 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505360128}
I20260709 13:13:50.921043   995 raft_consensus.cc:2368] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:13:50.925684  3766 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.927888   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:50.928341  3898 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:13:50.929251   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:50.929528  3746 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.930564   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1613) took 1529 ms. Trace:
I20260709 13:13:50.932364  3746 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:50.931025   998 rpcz_store.cc:276] 0709 13:13:49.401315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.401439 (+   124us) service_pool.cc:224] Handling call
0709 13:13:49.401779 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.054112 (+652333us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b4800210cd8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:50.202699 (+148587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.202728 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:50.207201 (+  4473us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.311956 (+104755us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:50.328252 (+ 16296us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.337627 (+  9375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.337706 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.781415 (+443709us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b18001340e0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa202a6 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1
0709 13:13:50.783352 (+  1937us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b18001340e0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:50.930203 (+146851us) raft_consensus.cc:1685] finished
0709 13:13:50.930227 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.930380 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9123,"prepare.queue_time_us":39396,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":154501,"raft.run_cpu_time_us":2511,"raft.run_wall_time_us":521295,"spinlock_wait_cycles":101742976,"thread_start_us":873742,"threads_started":2,"wal-append.queue_time_us":133547}
I20260709 13:13:50.933915   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.934167  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:13:50.934746   998 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:50.935037  1002 raft_consensus.cc:606] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:50.935137   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 70218.7us
I20260709 13:13:50.936015  1000 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:50.939867  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:50.940730  1595 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:50.907588   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:50.923244   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:50.943281  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:50.913612  3722 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:50.952112  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 43643.3us
I20260709 13:13:50.952998  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:50.956853   999 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:50.891680  3778 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:50.935508  1002 raft_consensus.cc:614] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:50.960122  1596 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:50.967845  1592 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
W20260709 13:13:50.983856   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.984884   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 118256us
W20260709 13:13:50.982187   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 53093.7us
I20260709 13:13:50.987262   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:50.975206  3796 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:13:50.999552   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 103798us
I20260709 13:13:51.000828  3920 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.002636  3717 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:51.010869  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.011955  1001 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:51.012450  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1740) took 1056 ms. Trace:
W20260709 13:13:51.017204  3677 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.705s	user 0.015s	sys 0.004s
I20260709 13:13:51.017789  3677 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.021165  3858 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.024761  3537 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:51.029503   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49442.1us
I20260709 13:13:51.030457  3922 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:51.030514  3857 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.031284  3922 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:51.019471   922 raft_consensus.cc:2368] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:51.012805  1001 rpcz_store.cc:276] 0709 13:13:49.962295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.699326 (+737031us) service_pool.cc:224] Handling call
0709 13:13:51.010655 (+311329us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b5800121598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:51.012380 (+  1725us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13052160}
I20260709 13:13:51.035256   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:50.977025  3834 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.032734  3844 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:13:51.040258  3920 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.056850  3844 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:51.057878  3920 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:51.051359   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.060287  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:51.059947   995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:13:51.003391  3717 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.048254  3922 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:51.072099  3858 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.061971   995 raft_consensus.cc:606] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:51.076327   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 134145us
I20260709 13:13:51.031895  3778 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.084774   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:51.088443   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 53282.9us
I20260709 13:13:51.085618   848 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.087899  3537 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.093058  3844 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.105201   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:51.109228   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 122063us
I20260709 13:13:51.094161  3858 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.114277  3857 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.123589  3855 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.143399  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.112083   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:51.138037   848 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:51.133397 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:51.143399   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:51.145449  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.145646  3834 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.142406   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:51.146366  3857 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:51.137501   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.163286  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.163515  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.178051   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:51.169349   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:51.184751   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 76831us
I20260709 13:13:51.188583   846 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.182704   851 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:51.163412  3857 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.196805   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:51.183738  1002 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:51.154191  3802 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:51.205641   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 62382.8us
I20260709 13:13:51.206836  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.161792  3844 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.223726   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.228365  1004 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:51.226603  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.228355   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1637) took 1134 ms. Trace:
I20260709 13:13:51.229389   999 rpcz_store.cc:276] 0709 13:13:50.088251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.220931 (+132680us) service_pool.cc:224] Handling call
0709 13:13:51.207171 (+986240us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54000898d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:51.222696 (+ 15525us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550731648}
I20260709 13:13:51.168524  3922 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.190982  3834 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.208720  3677 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.245786   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:51.246681   849 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.247534  3922 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.251585   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:51.233978  3537 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.212636  3864 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.266603  3864 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:51.258487   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.259637   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.259737  3717 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:51.260411  3923 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 13:13:51.260790  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:51.270691  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:51.271929  1567 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:51.272722  3493 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:51.273401  3493 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:51.273991  3493 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:51.274291  3493 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.274602  3493 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:51.261543  3537 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.277385  3537 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.281088  3858 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:51.282799  3921 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.281s	user 0.004s	sys 0.001s
W20260709 13:13:51.283900   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 53307.8us
I20260709 13:13:51.264221  3857 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.292919  3746 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:51.276299  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:51.295318  3913 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.299381  3844 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.305336   849 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:13:51.309854   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 206957us
W20260709 13:13:51.311043   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54622.6us
I20260709 13:13:51.322278  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1621) took 1263 ms. Trace:
I20260709 13:13:51.322736  1004 rpcz_store.cc:276] 0709 13:13:50.059131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.825270 (+766139us) service_pool.cc:224] Handling call
0709 13:13:50.825611 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.228846 (+403235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.322025 (+ 93179us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54001dbd54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:51.322215 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925795712}
I20260709 13:13:51.326330   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.327342   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.328172   929 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:51.257347  3920 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:51.328922  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.331199   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.318087  3718 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.352717   849 pending_rounds.cc:85] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:51.353072   849 pending_rounds.cc:107] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:51.343554  3855 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:51.354943  3834 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:51.354471  3607 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.354779   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.355401  3855 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.357853   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.358071  3855 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.359583   821 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.359905  3772 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:51.360960  3746 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:51.361042   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.361898  3746 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.362200  3766 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.362562  3805 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:51.362990  3855 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.363188  3805 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.365433  3805 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.366590  3746 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.375114   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1790) took 1128 ms. Trace:
W20260709 13:13:51.379098   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 55297.5us
I20260709 13:13:51.376950   922 rpcz_store.cc:276] 0709 13:13:50.215200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.215315 (+   115us) service_pool.cc:224] Handling call
0709 13:13:51.325001 (+1109686us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54004470d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:51.344025 (+ 19024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846128384}
I20260709 13:13:51.354952   849 raft_consensus.cc:2889] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.378607 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:51.387589   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.333753  3920 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.381218  3607 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:51.404207   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:51.405200   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.350836  3912 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:51.406741  3912 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:51.407150  3912 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.411671  3607 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.378072  3834 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.413733  3834 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:51.415946  3932 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
I20260709 13:13:51.418310   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.420017  3844 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:51.420701  3844 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:51.421984  3802 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.421813  3844 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.424448   847 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:51.426764   851 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:51.436477  3929 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:13:51.405130   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:51.443980   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 58623us
I20260709 13:13:51.454166   846 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:51.465054  3802 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:51.454771  3920 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.488984  3926 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:51.493376   996 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:51.493716   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:51.504796   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.493564   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.516732   999 raft_consensus.cc:2393] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:51.495546  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.497074   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.498860   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.500800  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.500384   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.504209  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1645) took 1155 ms. Trace:
I20260709 13:13:51.481298   851 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:51.495232  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.494928  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.499286   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.508304   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.519114  1003 rpcz_store.cc:276] 0709 13:13:50.348865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.839788 (+490923us) service_pool.cc:224] Handling call
0709 13:13:50.842029 (+  2241us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:50.842626 (+   597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.842662 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:50.843332 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 13:13:50.881095 (+ 37763us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:50.957696 (+ 76601us) log.cc:844] Serialized 14 byte log entry
0709 13:13:51.022746 (+ 65050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.022842 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.282247 (+259405us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b240023f438. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff8d89c4 00007f0cff8d6acb 00007f0cffa91dc9 00007f0cffa91a55 00007f0cffa9109d 00007f0d026c310e 00007f0d026da88c 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221
0709 13:13:51.503900 (+221653us) raft_consensus.cc:1685] finished
0709 13:13:51.503997 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.504134 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64706,"prepare.queue_time_us":89979,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":6904,"raft.queue_time_us":71215,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":1660,"spinlock_wait_cycles":75466496,"thread_start_us":37292,"threads_started":1,"wal-append.queue_time_us":134937}
I20260709 13:13:51.491967   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:51.540640   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79125.7us
I20260709 13:13:51.542001   969 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.540212   998 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:51.554018   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.554852   999 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:51.568343  1000 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.569221   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1646) took 1220 ms. Trace:
W20260709 13:13:51.569334  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 68694us
I20260709 13:13:51.488044  3677 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.577181   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85290us
I20260709 13:13:51.569962   997 rpcz_store.cc:276] 0709 13:13:50.349158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.851592 (+502434us) service_pool.cc:224] Handling call
0709 13:13:50.878304 (+ 26712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.963972 (+ 85668us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:51.158805 (+194833us) log.cc:844] Serialized 14 byte log entry
0709 13:13:51.568712 (+409907us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:51.568792 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:51.568861 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.568955 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.569129 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4498,"prepare.queue_time_us":23557,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":79,"thread_start_us":491599,"threads_started":2}
W20260709 13:13:51.571772  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 206608us
W20260709 13:13:51.572759   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.571338  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.506083  3772 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:51.489256  3765 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:13:51.551719  3746 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.588307  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:51.598145  1003 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:51.571867  3722 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:51.596695   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.600348   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.587651  3852 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.606142  3852 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.607141  3493 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.607973   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.608371  3852 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.609551  3765 consensus_queue.cc:1243] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:51.610102  3765 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:51.610548  3765 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:51.609892   753 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:51.611506  3765 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.611652   753 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:51.612928   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:51.610307   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 102110us
I20260709 13:13:51.617735  3803 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:51.618026   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:51.618208  3803 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:51.618707  3803 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:51.618883   923 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:51.619923  3843 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.617110  3842 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.620903  3842 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:51.621852  3842 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.623476  3766 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:51.624601  3855 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:51.625087  3855 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:51.625967  3855 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.627288  3855 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.619427   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.628119   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.628506  3493 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.628969   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.599906  3677 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.618212  3765 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:51.632238   998 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:13:51.653586  2959 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.636045  3911 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.639021  1000 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:51.613427   753 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 3, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:51.584157  3802 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:51.649513   929 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.633890  3912 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.656924   848 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:51.664016   921 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:51.666600   928 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:51.677017   999 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:51.662679  3843 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.691499   925 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:51.697326   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.698282   928 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.708250  3832 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:51.709939   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 222) took 10485 ms. Trace:
I20260709 13:13:51.710482   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.670640  3802 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:51.716279   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:51.720136   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 359160us
W20260709 13:13:51.723213   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 103890us
I20260709 13:13:51.721483   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.731375   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.743846   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.731981   821 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.732803   924 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:51.750710   926 raft_consensus.cc:2368] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:51.750597   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.768472  3940 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.770816   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.768479   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.773491   968 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.774489  3781 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:51.774938  3781 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.781234   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.787716   930 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:13:51.788254   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:51.790951   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.742036  3843 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:51.794574   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.793097  3864 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:51.800035   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.797748   929 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:51.801003   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:51.810055   927 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:51.818078   926 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:51.727619  3926 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:13:51.784595  3940 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.783218  3746 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.789091  3852 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:51.827737   924 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:51.791038  3844 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.795127  3843 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.827026   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.801261  3858 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:51.843055   927 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:51.801101  3864 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.798801  3607 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.845121   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:51.821910  3771 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.845849   852 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.845203  3429 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:51.851905  3937 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:13:51.845206  3607 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.854204  3922 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:51.854676  3922 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.856356  3607 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:51.857457  3922 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.858937  3922 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.859721  3802 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.862658  3781 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.863705  3940 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.864490  3677 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.845225   753 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:51.865943  3940 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.867748   852 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
W20260709 13:13:51.874699   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 142791us
I20260709 13:13:51.868074   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:51.869084  3912 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.828223  3493 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:51.800631  3772 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:13:51.891927  3493 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.892313  3772 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:51.892709  3772 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:51.893787  3772 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.877409  3429 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.895354  3429 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.897261  3864 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.873064   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.878533  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:51.879806   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.877599  3939 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:13:51.902585  3939 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:51.903024  3939 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:51.844553  3843 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:51.904095  3939 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.905263  3796 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:51.859715  3746 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:13:51.906780  3852 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:51.907227  3852 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:51.908043  3852 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.905818  3796 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:51.909407   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.910036  3796 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:51.912160  3844 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.889189   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:51.891390  3946 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.088s	user 0.001s	sys 0.001s
I20260709 13:13:51.798348  3833 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:13:51.920670   753 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:51.894157  3493 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:51.882208  3537 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:51.926123   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48617.1us
I20260709 13:13:51.900271  3772 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:51.874867  3858 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:51.905767  3939 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.889377  3920 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.913667   924 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:51.920524   925 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.928515   928 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:51.921759   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:51.710391   750 rpcz_store.cc:276] 0709 13:13:41.224146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.225457 (+  1311us) service_pool.cc:224] Handling call
0709 13:13:51.709696 (+10484239us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:49.539660 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:49.803849 (+264189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:49.803885 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:13:49.803902 (+    17us) tablet.cc:662] Decoding operations
0709 13:13:49.889146 (+ 85244us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:49.894436 (+  5290us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:49.896498 (+  2062us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:13:50.056290 (+159792us) tablet.cc:701] Row locks acquired
0709 13:13:50.056306 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:13:50.056517 (+   211us) write_op.cc:270] Start()
0709 13:13:50.056636 (+   119us) write_op.cc:276] Timestamp: P: 1783602830056466 usec, L: 0
0709 13:13:50.056655 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:13:50.057385 (+   730us) log.cc:844] Serialized 7497 byte log entry
0709 13:13:50.365662 (+308277us) op_driver.cc:464] REPLICATION: finished
0709 13:13:50.446499 (+ 80837us) write_op.cc:301] APPLY: starting
0709 13:13:50.446590 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:50.447134 (+   544us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:50.447303 (+   169us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:50.449179 (+  1876us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:50.449377 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 13:13:50.449406 (+    29us) write_op.cc:312] APPLY: finished
0709 13:13:50.449664 (+   258us) log.cc:844] Serialized 132 byte log entry
0709 13:13:50.449812 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:50.449928 (+   116us) write_op.cc:454] Released schema lock
0709 13:13:50.449965 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:50.450257 (+   292us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:50.450275 (+    18us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36904,"reported_tablets":49,"spinlock_wait_cycles":7936,"child_traces":[["op",{"apply.queue_time_us":80594,"apply.run_cpu_time_us":3746,"apply.run_wall_time_us":3747,"num_ops":19,"prepare.queue_time_us":233708,"prepare.run_cpu_time_us":50700,"prepare.run_wall_time_us":616596,"raft.queue_time_us":139046,"raft.run_cpu_time_us":10718,"raft.run_wall_time_us":75180,"replication_time_us":308873,"spinlock_wait_cycles":78080,"thread_start_us":342635,"threads_started":4,"wal-append.queue_time_us":164612,"wal-append.run_cpu_time_us":46707,"wal-append.run_wall_time_us":1228402}]]}
I20260709 13:13:51.928901  3607 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:13:51.935822  3833 consensus_queue.cc:1243] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:51.940212   847 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:51.943095  3940 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:51.901261   927 raft_consensus.cc:2393] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:13:51.966976   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:51.966768  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 351915us
I20260709 13:13:51.984480  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.979164  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:51.989089  3537 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.985504  1002 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:51.993861  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:51.994614  3765 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:51.995203  3765 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:51.995963  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:51.997320  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:51.997908  3765 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:51.998220  3604 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.999058  3604 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:52.000370  3604 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:52.000790  3604 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.001338  3604 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:52.003964  2959 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:52.004340  2959 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:52.005280  3920 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.005196  3771 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:52.005995   921 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:52.005103  2959 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.008024  2959 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.008833   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.013413   846 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:51.978894  3803 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.017371  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:52.017949  3498 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:52.018352  1599 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:13:52.019205  3803 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:52.019711   999 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:52.020020   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54265.3us
I20260709 13:13:52.020877   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.022215   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:52.024454  3855 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:52.024878  3855 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:52.025209  3833 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.025856  3858 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.029954   753 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:52.032420   925 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:52.037177  3604 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.025230  3855 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.029423  3537 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:52.082358   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.106423   753 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:13:52.111366   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 337970us
I20260709 13:13:52.124033  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:52.124318  3952 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:13:52.133401   922 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:52.137094   999 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:52.138046   753 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:52.144531   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.144250   753 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:52.145998   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 345620us
I20260709 13:13:52.150724   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.153623   753 catalog_manager.cc:5697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:52.161800   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 283290us
I20260709 13:13:52.164613   818 leader_election.cc:400] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:52.165069   818 leader_election.cc:403] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:52.162562   753 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:52.173641   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.158771  3832 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:52.180979   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:52.185601   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.186867   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.189759   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.190114  2959 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:52.190618  2959 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.191047  2959 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.183131   753 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:52.200187   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:52.229044   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 53140us
I20260709 13:13:52.235574  1003 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:13:52.235945   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.176815  3604 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.235317  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:52.240779  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:52.241370  1001 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:52.180904  3832 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.229970   753 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:52.239804   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:52.247429   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 142265us
W20260709 13:13:52.248813   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 48746.9us
I20260709 13:13:52.251677   753 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 4 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:52.252506   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.259552   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.261751  2959 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.263701  2959 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.252918   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.266418   921 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:52.235688  3940 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:52.272462   753 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 3 to 5, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 5 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:52.270156   999 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:52.198707  3766 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:52.281276  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:52.281962  1592 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:52.282827  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:52.283398  1600 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.245499  3833 raft_consensus.cc:2764] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:13:52.246431  3832 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.296612  1600 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
W20260709 13:13:52.300078   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 277922us
I20260709 13:13:52.302501   821 leader_election.cc:400] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:13:52.303071   821 leader_election.cc:403] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:13:52.304728  3843 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.306061  1000 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:13:52.304786   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.296679  3953 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:52.184310  3429 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:13:52.253283  3842 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.333285  3498 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.328408   753 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 3 to 4, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:52.364719  2959 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:13:52.372041   753 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:52.381927  3852 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:52.384573   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82279.5us
I20260709 13:13:52.386590   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.387584  3959 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:52.387986  3959 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:52.388330  3959 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.395705  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:13:52.408656   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 178900us
I20260709 13:13:52.416599  3722 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:52.417197  3722 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:52.418252   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:52.418114  3722 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.429910   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.430733  3802 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:52.435050  1001 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:52.375804  3766 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:52.444211  3960 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:52.445683  3960 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.446218  3960 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.437999  3781 raft_consensus.cc:993] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:13:52.450918   849 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:52.448290   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:52.392541  3498 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:52.454384   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.454953   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:52.484608   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49740.2us
I20260709 13:13:52.491720  3766 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:52.493268  3968 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
W20260709 13:13:52.495947   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46601.2us
I20260709 13:13:52.432562  3855 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.453589  3803 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:52.497213 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:52.450928  3604 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:52.523487   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 131224us
I20260709 13:13:52.445704  3925 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:13:52.511503  3843 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:52.535460   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.539348   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.540547   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.541147   928 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:52.541703   930 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:52.542301   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.513823  3803 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.548599   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.549129   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.550217  3607 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:52.550480   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.552286   922 raft_consensus.cc:2376] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7. Re-sending same reply.
I20260709 13:13:52.552560   926 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:52.559233   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.560004   928 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.564971   923 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:52.531658  3956 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:52.550612  3803 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:52.547152  3979 raft_consensus.cc:1081] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
W20260709 13:13:52.584455   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50608.8us
I20260709 13:13:52.527163  3604 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.550755  3607 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:52.586920   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 138684us
I20260709 13:13:52.588089   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.589810   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.592089   926 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:52.561218  3802 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:52.596194   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.599794  3986 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:52.600492  3986 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:52.601809  3986 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:13:52.602197  3971 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.247s	user 0.002s	sys 0.000s
I20260709 13:13:52.607141  1003 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:52.609201   999 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:52.610235  1001 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:52.585887  3956 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.593062  3781 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:52.615193  3498 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:52.592484  3717 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:52.619164   922 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:52.625547  1003 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:52.625852   922 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:52.623595   928 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:52.592190  3843 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:52.627357  3985 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:52.641503   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.642421   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:52.644922   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45871.3us
W20260709 13:13:52.646662  3974 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:13:52.596421  3802 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.651816   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.625267  3926 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.616763  3718 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:13:52.616019  3962 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:13:52.661058   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.662909   926 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:52.644176  3717 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:52.664217   930 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:13:52.658283  3926 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.672253  3976 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:13:52.673543  3940 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:13:52.682243   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:52.688764   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:52.644678  3781 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.663197  3855 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.723255   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:52.716333  3498 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:13:52.713507  3940 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:52.690366  3987 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:52.737282   821 leader_election.cc:400] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:13:52.726661  3985 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:52.663539  3607 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:52.748984   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.780164   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.736533  3855 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:52.737406  3940 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:52.739928  3604 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.742208  3843 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:13:52.779712   923 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42382
I20260709 13:13:52.792202   923 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:52.792631   923 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:52.793743  3717 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:52.795665  3978 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:13:52.783787  3855 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:52.796262  3855 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:52.747102  3987 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:52.800944  3987 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.786830  3852 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:52.806679  3803 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.786442  3940 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.746376  3985 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.788512  3960 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:52.807801   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:52.821194   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.838347   923 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.839095  3852 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:52.842371  3832 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.558s	user 0.004s	sys 0.004s
I20260709 13:13:52.843658  3781 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:52.843699  1599 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:52.844792  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:52.845897  1600 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:52.846092  3959 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.839036  3855 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.864719  3959 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.846882  1592 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:52.865792  3855 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:52.847723  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:52.823246  3985 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:52.866400  3940 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.868232   923 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.850449  1597 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:52.875192  3855 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.876022  3855 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.876312  3985 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.878659  3985 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:52.879383  3855 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:52.881279  3803 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:52.882743  3926 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.884378  3802 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:52.887424   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 79732.7us
I20260709 13:13:52.889853  3855 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.890623  3855 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.891824  3832 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:52.892138   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.892894   846 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:52.893975  3832 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.894006   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:52.898445  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53718.4us
W20260709 13:13:52.898782   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 254038us
I20260709 13:13:52.899590  3857 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:52.900772  3855 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:52.901130   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.901481  3913 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:52.901813   849 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:52.913340   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.914898   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:52.916764   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:52.917850  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:52.919911  1595 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:52.917389  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:52.921418  1597 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:52.921954   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.923209  3857 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:52.923241  3976 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:52.925228  3857 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:52.926077  3961 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:52.926156  3857 consensus_queue.cc:1243] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:52.926753  3961 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:52.926667  3858 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:52.928995   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:52.929860  3772 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:52.930249  3772 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.930603  3772 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:52.931438   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.931838   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:52.909171  3832 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.945528   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.949270   846 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:52.954357   851 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:52.955718   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.957065   850 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:52.958117   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.958423   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.959008   853 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:13:52.958119   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.962028   851 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:52.963008   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.963268  3604 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:52.964119  3604 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:52.967083   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.983134   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:52.991676   846 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:52.997710   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:52.998790   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.001108   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.002569   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:53.003809   850 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:53.007558   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:53.008100   997 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:53.008275   997 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:53.022971   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.010689   997 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.025751   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:52.997964  3832 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.029186   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.030645   997 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:53.036108   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.037039   855 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:53.043970  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:53.046607   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.048753   846 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:53.012235  3940 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:53.052356   846 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:13:53.063449   846 pending_rounds.cc:85] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:13:53.065054   846 pending_rounds.cc:107] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:53.065140   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.065392   846 raft_consensus.cc:2889] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:53.065786   852 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:53.067310   851 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:53.045080   922 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.069017   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:53.085117   922 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:53.085638   996 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:53.086959   996 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:53.088059   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.089634   996 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:53.090837   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.094738   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.095896   997 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:13:53.097990   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.100258   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.100868   846 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:53.112793   852 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:53.113991   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.115128   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.116199   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.117219   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.128971   852 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:53.134884   922 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:53.138622  3604 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:53.140709   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.146801   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:53.147889  3967 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.814s	user 0.006s	sys 0.003s
W20260709 13:13:53.150017  4001 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:53.151278   852 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:13:53.151811  3429 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 13:13:53.152686  4000 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:13:53.156494   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:53.157593   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.158923  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1663) took 1038 ms. Trace:
W20260709 13:13:53.163775  4004 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:13:53.164294  4007 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:13:53.169687  4003 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:13:53.173820   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.174006   849 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:53.174917  3956 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:53.180224   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:53.183463   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:53.321171   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.355000   925 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:53.288609   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:53.358129   846 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:13:53.358590   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:53.358757   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:53.350656   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.360354   855 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:53.362637  1566 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:53.288250   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:53.365260   924 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:53.126809  3852 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:53.287217   926 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:53.359247   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.244091   854 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:13:53.197624  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55367.7us
I20260709 13:13:53.370492  1000 rpcz_store.cc:276] 0709 13:13:52.120819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.123720 (+  2901us) service_pool.cc:224] Handling call
0709 13:13:53.158739 (+1035019us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540034e9d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:53.158873 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416594304}
I20260709 13:13:53.371306   846 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:53.359530   929 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:53.208267  2959 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:53.372846   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.373420   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.373553   846 raft_consensus.cc:2376] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6. Re-sending same reply.
I20260709 13:13:53.374488   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.374583   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:13:53.374825   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.378132   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.375253   846 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:53.372354  2959 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.378950   854 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:53.380436   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.381628   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.354707  3983 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.354727  3498 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.387682   854 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:53.396572   846 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:53.243508  3604 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.414549   853 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:53.416190   929 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:53.418612   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.419231   846 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:53.424464   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.433269  2959 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:53.426716  3832 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:53.403168  3855 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:53.441062   846 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:13:53.442330   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 83670.2us
I20260709 13:13:53.437852  3604 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.446019  3604 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:53.446727  3855 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.452272  3855 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:53.449613   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:53.456650  3772 consensus_meta.cc:287] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:13:53.457247  3772 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.459820  3990 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:53.461166   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 200831us
I20260709 13:13:53.461449  3982 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.462214  3982 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.462814   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:53.470206  3844 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:53.470973  3842 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:53.485168  3993 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.533s	user 0.002s	sys 0.000s
I20260709 13:13:53.484817  3842 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.484848  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:53.472513  3771 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:53.472692  3955 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:53.489583  3955 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.489516  3771 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.484033  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:53.492542  4006 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
I20260709 13:13:53.491918  3844 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:53.494629  4015 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:13:53.496492   852 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:53.499025  3969 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.140s	user 0.001s	sys 0.012s
I20260709 13:13:53.499778  3956 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:53.500192  3956 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:53.500566  3956 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:53.500978   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1913) took 1491 ms. Trace:
I20260709 13:13:53.501430   847 rpcz_store.cc:276] 0709 13:13:52.009471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.009634 (+   163us) service_pool.cc:224] Handling call
0709 13:13:52.010062 (+   428us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:52.010481 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.010520 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:52.011217 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 13:13:52.028369 (+ 17152us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4800002dd8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:52.161504 (+133135us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:52.180520 (+ 19016us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.198985 (+ 18465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.199056 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:53.438768 (+1239712us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:53.500692 (+ 61924us) raft_consensus.cc:1685] finished
0709 13:13:53.500717 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.500878 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18208,"prepare.queue_time_us":34695,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":55522,"raft.run_cpu_time_us":1155,"raft.run_wall_time_us":36416,"spinlock_wait_cycles":57424000,"thread_start_us":149852,"threads_started":1,"wal-append.queue_time_us":281031}
I20260709 13:13:53.508772   850 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:53.515245   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.518014   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.519742   924 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:13:53.519892  3960 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:53.528973  3960 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.417979  3498 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.506850  3842 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:53.520607  3920 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:53.432174  3983 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.604993  4019 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:53.559484  3855 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:53.530944  3429 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:13:53.611756  3941 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
W20260709 13:13:53.746003   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.844340  1595 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:53.775048  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 399450us
W20260709 13:13:53.856020   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 297279us
W20260709 13:13:53.760458   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:53.858968   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.632469  3842 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:53.859531   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.861397   925 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:53.858505  3689 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:13:53.700130  3855 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.831521  3956 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.632875  3844 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:53.849992  3802 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:13:53.874810  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:53.865001  3960 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:53.866312   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:53.867492  3983 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:53.868727  3604 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:53.832531  3982 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:53.882064   996 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:53.872654   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.719115  3987 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:53.822630  4011 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:53.846141  3771 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:53.651196  3967 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:13:53.700881  3986 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:53.872169   846 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:53.875739  1004 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:13:53.631965  3920 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:53.883037   848 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:13:54.095876   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1656) took 1116 ms. Trace:
I20260709 13:13:53.951905 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:53.940557  2959 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:54.101864   853 rpcz_store.cc:276] 0709 13:13:52.964472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.965676 (+  1204us) service_pool.cc:224] Handling call
0709 13:13:54.076149 (+1110473us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54002b8254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:54.081037 (+  4888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944818432}
I20260709 13:13:54.068116   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.073163  3802 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.014350  3956 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:13:53.940878  3832 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:53.883246  3604 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:53.956355  3855 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:53.940671  3852 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:53.881067  3537 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.227s
W20260709 13:13:53.939724  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.028s	sys 0.046s
W20260709 13:13:54.094385  4002 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 1.047s	user 0.005s	sys 0.004s
W20260709 13:13:54.057823  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 192509us
I20260709 13:13:54.113802   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.114297   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:54.113040  3987 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.117452   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.119076  3920 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.123183  3970 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:54.111869  3982 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.132334   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.149652  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:54.132473  2959 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:54.138072  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.035s	sys 0.049s
I20260709 13:13:53.886404   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:53.958706  3498 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.144456   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.157735   846 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 3.
I20260709 13:13:54.143011   848 raft_consensus.cc:2376] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7. Re-sending same reply.
I20260709 13:13:53.873961  3842 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.147337   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1745) took 1495 ms. Trace:
I20260709 13:13:54.148154   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.153491  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:54.160240  1004 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:54.164425   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.164764  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.165108   848 raft_consensus.cc:2376] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5. Re-sending same reply.
I20260709 13:13:54.166154   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.166617   848 raft_consensus.cc:2376] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8. Re-sending same reply.
I20260709 13:13:54.167230   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.168166   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.168694   846 raft_consensus.cc:2376] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4. Re-sending same reply.
I20260709 13:13:54.169426   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.148268  3998 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.159591   928 rpcz_store.cc:276] 0709 13:13:52.646934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.648287 (+  1353us) service_pool.cc:224] Handling call
0709 13:13:52.648638 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.724609 (+ 75971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.724653 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.729748 (+  5095us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.780817 (+ 51069us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:52.813157 (+ 32340us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.813392 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.180967 (+367575us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b1800438220. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:54.142476 (+961509us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400175a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:54.142529 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.142559 (+    30us) raft_consensus.cc:1685] finished
0709 13:13:54.142566 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.142648 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43664,"prepare.queue_time_us":40728,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":10898,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":4487,"spinlock_wait_cycles":170761472,"thread_start_us":50553,"threads_started":1,"wal-append.queue_time_us":94935,"wal-append.run_cpu_time_us":2534,"wal-append.run_wall_time_us":1134163}
I20260709 13:13:54.177121   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.177667   846 raft_consensus.cc:2376] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7. Re-sending same reply.
I20260709 13:13:54.112787  4011 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.174945   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.175978  2959 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.179296   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.147375  3956 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:54.196162   851 raft_consensus.cc:2376] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10. Re-sending same reply.
I20260709 13:13:54.196405   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.196854  2959 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:54.133257  3986 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.198236   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1938) took 1007 ms. Trace:
I20260709 13:13:54.197616   853 raft_consensus.cc:2376] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7. Re-sending same reply.
I20260709 13:13:54.141606  3802 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.208834  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.209537  1592 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:13:54.198552  2959 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.165421  4021 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.146358  3832 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.212395   848 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:54.180167  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:54.203665   846 rpcz_store.cc:276] 0709 13:13:53.190867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.197450 (+1006583us) service_pool.cc:224] Handling call
0709 13:13:54.197807 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.197969 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.197990 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.198026 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:54.198053 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.198104 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.198190 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.146641  3852 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.217360   851 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:54.156924  3982 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.218923   846 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:54.154179  3970 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.229022   855 raft_consensus.cc:2410] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:54.229909   851 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:54.231683  2959 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.232195   848 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:54.213097  3717 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.245278   851 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:54.247403   851 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:54.195767  3998 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.258213   851 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:54.211476  3956 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.273687  4039 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:54.274834  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 94722.8us
I20260709 13:13:54.234099  3842 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.289359   851 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:13:54.276593   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 103644us
I20260709 13:13:54.257097   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1936) took 1066 ms. Trace:
I20260709 13:13:54.290838   852 rpcz_store.cc:276] 0709 13:13:53.190292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.497198 (+306906us) service_pool.cc:224] Handling call
0709 13:13:53.497463 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.497640 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:53.878500 (+380860us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.256628 (+378128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.256706 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:54.256771 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.256861 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.257023 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49627,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":467,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":23750,"thread_start_us":757745,"threads_started":2,"wal-append.queue_time_us":381565}
I20260709 13:13:54.218031  4021 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:54.302655  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 134854us
I20260709 13:13:54.247370  3717 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.292496  3429 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:54.313038   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.300086  3941 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:54.307334  3781 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:54.327873  3987 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.407543  4011 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.409013  4021 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.409704  3970 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:54.438845   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.479338   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.482615   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 163708us
I20260709 13:13:54.489589   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:54.490398   928 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:13:54.492750   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:54.331722  3429 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.404035  4043 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:54.406437  3986 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.389744  3781 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.418598  3802 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.475569  3855 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:54.524039  1592 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:54.524084   896 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:54.509677  3832 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.510049  4035 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:54.522583  3604 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.514424  4036 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:13:54.526571  4043 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.606604  4043 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.527081  3855 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.609624  3855 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:54.589532  4031 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:13:54.613168  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1679) took 1704 ms. Trace:
I20260709 13:13:54.613535  1001 rpcz_store.cc:276] 0709 13:13:52.909020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.909187 (+   167us) service_pool.cc:224] Handling call
0709 13:13:52.917898 (+  8711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:53.245735 (+327837us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:53.333195 (+ 87460us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.095252 (+762057us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800349e40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df23a 00007f0d02b49a2a
0709 13:13:54.292088 (+196836us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.307034 (+ 14946us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.307190 (+   156us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:54.307221 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.307295 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.612765 (+305470us) raft_consensus.cc:1685] finished
0709 13:13:54.612894 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.613059 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350308,"prepare.queue_time_us":87425,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2045,"raft.queue_time_us":322739,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":17129,"spinlock_wait_cycles":23951744,"thread_start_us":196566,"threads_started":1,"wal-append.queue_time_us":960639}
I20260709 13:13:54.590361  3852 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.590476  3832 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:54.591188  3970 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:54.535939  3956 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.607404  4043 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:13:54.592628  3429 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.533921  3858 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:54.535820  4035 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.543373  3998 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.663050  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:54.663574   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.540319  3677 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:13:54.671105   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.672253   852 raft_consensus.cc:2376] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4. Re-sending same reply.
I20260709 13:13:54.668571   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:54.673512   851 raft_consensus.cc:2376] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10. Re-sending same reply.
I20260709 13:13:54.680387  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.681792  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.688720   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:54.670035   895 connection.cc:256] RPC call timeout handler was delayed by -1.1549777929999436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 13:13:54.696110   895 connection.cc:256] RPC call timeout handler was delayed by -1.1531777929999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 13:13:54.677500  3855 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.696923   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.623813  3970 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:54.697558   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.624275  3832 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.680725  3852 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:54.706079   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.710597   852 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:54.671231  4041 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:13:54.713790  4037 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.678141  4047 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:13:54.716102   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.717829   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:54.697211  4055 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:13:54.727630   850 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:54.707461  3717 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.735831  3956 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:54.736555  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:54.748553  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:54.753845  1004 raft_consensus.cc:2410] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:13:54.709227  3855 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.751276   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:54.709152  4026 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:54.752302   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:54.759478   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 220099us
W20260709 13:13:54.764742   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.768205  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:54.719902  3852 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.758409  4055 consensus_queue.cc:1243] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:54.757647  4052 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:54.724958  4037 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.761009  4034 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:54.782963   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 245887us
I20260709 13:13:54.783797   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:54.757373  3956 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.784518   847 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:54.785199   921 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: [6.6-7.7]
I20260709 13:13:54.772523  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:54.786398  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.788988   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1671) took 1429 ms. Trace:
I20260709 13:13:54.789398   846 rpcz_store.cc:276] 0709 13:13:53.360127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.214837 (+854710us) service_pool.cc:224] Handling call
0709 13:13:54.218720 (+  3883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.221786 (+  3066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.788799 (+567013us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b540041f0d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:54.788930 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8183680}
I20260709 13:13:54.795424  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.803761  4035 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:54.804891   855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1677) took 1442 ms (client timeout 1660 ms). Trace:
I20260709 13:13:54.804730  3429 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:13:54.805253   855 rpcz_store.cc:269] 0709 13:13:53.362066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.067767 (+705701us) service_pool.cc:224] Handling call
0709 13:13:54.804674 (+736907us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540043e754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:54.804820 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278322560}
I20260709 13:13:54.764142  4026 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.807497   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1941) took 1309 ms. Trace:
I20260709 13:13:54.807909   848 rpcz_store.cc:276] 0709 13:13:53.497739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.240644 (+742905us) service_pool.cc:224] Handling call
0709 13:13:54.806187 (+565543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.806376 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.806407 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.806449 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:54.806476 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.806532 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.807342 (+   810us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b540043e750. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:54.807441 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299917056}
I20260709 13:13:54.809341  4026 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.810751  3772 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.784536  4034 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.796380  4019 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:54.810660  3998 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:54.795833  4041 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:54.813001  3998 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:54.813887  3998 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.818801  3961 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.820302  4013 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:54.820873  4013 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:54.822126  4013 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.803808   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1939) took 1544 ms. Trace:
I20260709 13:13:54.827109   853 rpcz_store.cc:276] 0709 13:13:53.259438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.206077 (+946639us) service_pool.cc:224] Handling call
0709 13:13:54.219228 (+ 13151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.258055 (+ 38827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.258083 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.258118 (+    35us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:54.258147 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.803579 (+545432us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b54004097d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:54.803646 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.803753 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88855424}
I20260709 13:13:54.830466   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 370) took 1730 ms. Trace:
W20260709 13:13:54.858654   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.830973   854 rpcz_store.cc:276] 0709 13:13:53.100278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.418751 (+318473us) service_pool.cc:224] Handling call
0709 13:13:53.419000 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.452914 (+ 33914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.452958 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.453016 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:53.453077 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.830221 (+1377144us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b540023b254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:54.830260 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.830392 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77444864}
I20260709 13:13:54.866910  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:54.823272  4013 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:54.871917  3986 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:54.877231  4011 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:54.877389  3842 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.904899  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.886415  3772 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.906131  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.825556  4024 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:13:54.895802  3986 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.903239  3803 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:54.950479  3803 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:54.951457  3803 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.952591  3803 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:54.904608  3832 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.920372  4056 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:54.929653  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.956492  4056 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:54.940502  3855 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:54.941666  3956 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.916683  3842 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.957456  4056 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:54.958912  4056 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:54.959936  3842 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:54.895834  4011 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:54.943809  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:54.960366  3852 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:54.930033  3970 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:54.979910   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 180038us
W20260709 13:13:54.997982   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 200183us
I20260709 13:13:54.922384  3772 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.005136  1597 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:55.006909  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 563) took 1503 ms. Trace:
I20260709 13:13:55.009615   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:55.010572   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 213768us
I20260709 13:13:54.976533  3855 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.008076  1597 rpcz_store.cc:276] 0709 13:13:53.503783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.504134 (+   351us) service_pool.cc:224] Handling call
0709 13:13:53.871319 (+367185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.997622 (+1126303us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b540063fc20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b411c0 00007f0d02b42231 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c
0709 13:13:55.005852 (+  8230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.006769 (+   917us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14227328}
I20260709 13:13:55.041314   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.104305  3604 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.105098  3604 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.021831  3855 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.106894  4011 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:55.114692  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.134119  4071 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:55.134912  4071 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:55.108371  3986 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:55.160954   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.107581  3604 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.109210  3772 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.108227  3855 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.160597  4035 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:55.169353  4035 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:55.211902  4035 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.222024  4071 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.279534  4035 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.281894  4035 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.287848  4066 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:55.290422   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 280865us
I20260709 13:13:55.309428   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:55.312094   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.320704   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:13:55.322551   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 281291us
I20260709 13:13:55.314568  4011 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:55.315872  3986 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.342578   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.344494   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:55.345654   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.322956   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.323334  3803 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:55.350483   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.350956   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.354391   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.354692   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:55.351891  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.352015   853 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:55.356407   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.357098   853 raft_consensus.cc:2393] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:55.320997  4066 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.357769   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.358464   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:55.358474   853 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:55.359311   852 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:55.359551   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.360037   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.360240   853 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:55.360158  4066 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:55.364071   852 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:55.365603   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.366482   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.351984   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.368443   848 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:55.368645   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.369550   927 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:55.380643   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:55.383157   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.346184  3772 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.348330  3986 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.406888   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.386476   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.383061   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.345358  4011 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.411960   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.390136   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.412799   924 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:13:55.412943   921 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:13:55.348315  3970 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:55.344794  3987 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:13:55.367328  3984 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:55.402370   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.348580  3677 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:55.324136  3802 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:13:55.347939  3843 consensus_queue.cc:1059] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:13:55.402954   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.403702   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.425022   922 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 3.
I20260709 13:13:55.425208   927 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:55.405717   926 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
W20260709 13:13:55.407958   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.346992  4065 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.408464   929 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:55.408893   930 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:55.410064  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.415655   925 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:55.433323  1000 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:55.438351   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.445573   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:55.446045  3998 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:55.447129  3998 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:55.447566  3998 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.445616  3986 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:55.448110  3998 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.449441   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:55.451823   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 108674us
I20260709 13:13:55.455214   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.369102  4035 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:13:55.405102  3832 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.462502   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.463344   997 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:13:55.456575  4079 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:13:55.534359   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 164613us
W20260709 13:13:55.534505   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 85156.9us
I20260709 13:13:55.537523   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.454322  3984 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:55.453608  3677 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:55.540588   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.426611  4076 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:55.571924   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.575314   995 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.574577   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.420027  3772 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.434899  3970 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.592926  3772 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.594261   849 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:55.540282  4073 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.580360  4075 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:55.472487  3832 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.602376  3677 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.574124  3803 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:55.603797  3803 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:55.603997  3832 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.604209   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.604938  4057 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:55.604666  3803 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.605720  3803 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.596935  3852 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.613685  4030 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.353s	user 0.010s	sys 0.017s
I20260709 13:13:55.614159  4071 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.617843   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.622217  3852 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.609426  3842 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:55.609556  3982 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:55.625089   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.630313   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:55.631470   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.631873   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.632421  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:55.635300  1600 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:55.636191  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:55.640795  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:55.641968   968 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:55.639947  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1692) took 1527 ms. Trace:
I20260709 13:13:55.645617  1002 rpcz_store.cc:276] 0709 13:13:54.112584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.112713 (+   129us) service_pool.cc:224] Handling call
0709 13:13:54.112952 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.196255 (+ 83303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.196302 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.196359 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:54.196403 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.639645 (+1443242us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b54000884d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:55.639693 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.639859 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190930816}
I20260709 13:13:55.628664  4075 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:55.675622  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:55.677477   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49952.5us
W20260709 13:13:55.694978   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.695307   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:55.697113  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.645053  3717 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.646116  3689 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.702471  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:55.645932  3842 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:55.713903  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.745620  1002 raft_consensus.cc:2393] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:55.745774  3842 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.746656  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.747901  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.748243   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.738724  3970 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.707587  4034 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:55.739506  4011 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:55.707643  4068 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:55.749713  4068 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.750196  4068 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:55.750335  4088 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:13:55.751475   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:55.752929  4090 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.753399  4090 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.748431  1002 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.760659   896 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.733314   995 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:13:55.749106   998 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:13:55.762171   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.763409   995 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:55.765425   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.770519   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:55.772372   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:55.755538  4090 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.776221  3772 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.778749   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.779362   995 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.779654   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.781315   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.785535   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.786928   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:55.788631   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:55.790515   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:55.793816   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 152584us
I20260709 13:13:55.795186   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.768759  3832 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:55.805691   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.629944  3984 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.764333  4075 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:55.790203  3858 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:55.824851  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.826136   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.794063  3772 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:55.791697  4090 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.793311  4089 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:55.793782  3855 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.830816   995 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:13:55.832907  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.833421   998 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
W20260709 13:13:55.854647   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 402177us
I20260709 13:13:55.809608  3864 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:55.813198  4062 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:55.807570  3852 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.819185  4072 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:55.764561  3689 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:55.857257   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 68720.5us
I20260709 13:13:55.836650  3832 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:55.827730  3858 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.858039   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.825851  3913 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:55.860342   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.826275  4056 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:55.836241  3772 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:55.842186  4089 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.852644  3855 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.819404  4094 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:13:55.861143  3429 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.882362   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.825573  4075 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.859870  4062 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:55.860432  3781 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:55.885152   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.874480  3832 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:55.879113  3858 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:55.883414  3772 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.887358  4089 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:55.884387  3913 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:55.903499   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.912590  1002 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:55.902086   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.877607  3864 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:55.922070   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 95973.8us
I20260709 13:13:55.893446  3855 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.929189   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.930630  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.933674   998 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
W20260709 13:13:55.937119  4097 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:55.914178  3955 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:55.940230  3864 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.917781  3832 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.912428  4062 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.945955  1002 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:55.946587   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:55.951437   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:55.953011   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52689us
I20260709 13:13:55.903712  4075 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.960913   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1689) took 1852 ms. Trace:
I20260709 13:13:55.962419  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.963575  4075 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:55.928143  3986 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:13:55.967594  4075 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:55.963596  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:55.969597   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:55.968865  4084 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.970191  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.962255   996 rpcz_store.cc:276] 0709 13:13:54.108590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.108787 (+   197us) service_pool.cc:224] Handling call
0709 13:13:54.117285 (+  8498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.219892 (+102607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.219922 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.219962 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:54.219992 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.953778 (+1733786us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400046ad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:55.960726 (+  6948us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.960860 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235529984}
I20260709 13:13:55.971544  1001 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.974440  4056 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:55.975020  3689 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:55.978456   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:13:55.983163  4054 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 1.281s	user 0.010s	sys 0.011s
I20260709 13:13:55.984475  3781 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:55.984900  3781 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:55.987057  1003 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:13:55.986467   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:55.989626  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 566) took 1831 ms. Trace:
I20260709 13:13:55.989981  1595 rpcz_store.cc:276] 0709 13:13:54.158395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.158549 (+   154us) service_pool.cc:224] Handling call
0709 13:13:54.158851 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.307824 (+148973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.307868 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.314354 (+  6486us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.527229 (+212875us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:54.527266 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.527331 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.982500 (+1455169us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:55.985322 (+  2822us) raft_consensus.cc:1685] finished
0709 13:13:55.985343 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.989570 (+  4227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25071,"raft.queue_time_us":87930,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":25950,"spinlock_wait_cycles":5085952,"thread_start_us":331553,"threads_started":2,"wal-append.queue_time_us":364893}
I20260709 13:13:55.995056   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.995975   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.994396   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:55.995941   850 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:13:55.997357   853 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:13:55.999006   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.999840   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.000478   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:56.001363   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:56.001637   999 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:13:56.002898   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.004482   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:56.004861   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:56.005671   850 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:56.005812   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:56.006172  4071 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:56.006809   850 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:56.007171  4101 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:13:56.007854   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:55.985895  3429 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:13:56.009347  3913 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.009624   848 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:56.010053  3842 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:55.989116  4076 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:55.987360  3781 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.011582  3986 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:56.015625  3998 consensus_meta.cc:287] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 13:13:55.986131  4084 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:55.962795  4068 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.017038   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.004987   853 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:56.005671   926 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:56.017598   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:56.029150   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.030501   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:56.030936   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:56.044983   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.047155  1001 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:13:56.049484   922 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:56.049721   967 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:56.050917   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:55.991307  4056 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:13:56.037982  3986 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.053201  3986 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.064606   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.065130  3832 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.065822   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.066421  4089 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.066864   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.067200  3832 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.068639  3858 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.069634   848 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:56.072736   849 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:56.024940  3842 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:56.084226   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66719.9us
I20260709 13:13:56.089569   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:56.096038   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:56.063181  3855 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.100951   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.101576   848 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:56.104074   924 raft_consensus.cc:2368] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9.
I20260709 13:13:56.080642  3689 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:56.111732  3689 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.113142  3772 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.092586  3803 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:56.113736  3961 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:56.114208  3803 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.114665  3803 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.039139  3998 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.092000  3842 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:56.084229  3984 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:56.109584  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:56.111145  4065 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:56.116220   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.119578   853 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:56.128808   999 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:56.083964  3913 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:56.131759   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.132684   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 385) took 1638 ms. Trace:
I20260709 13:13:56.133569   925 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:56.133054   928 rpcz_store.cc:276] 0709 13:13:54.494307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.494835 (+   528us) service_pool.cc:224] Handling call
0709 13:13:54.495154 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.616752 (+121598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.616784 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.616827 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:54.616870 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.132459 (+1515589us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54002314d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:56.132517 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.132632 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279163648}
I20260709 13:13:56.133987  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:56.147912   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.148950   928 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:56.150354   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.113922  3689 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:56.202301   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.104636  4066 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:13:56.125387  3982 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:56.376322  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 285078us
W20260709 13:13:56.429965  4113 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260709 13:13:56.432189   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:56.433045   848 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:13:56.269877   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:56.436301   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.436957   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.233487   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:56.286891  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.130482  4065 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:56.540624   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.403136  1004 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:56.426529  4089 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:56.130297  3955 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:56.374269  1003 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:56.575387   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.522769  4062 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.614799  4062 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.620805   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.622320   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.622965   926 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:56.215029  3689 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.330s
W20260709 13:13:56.626190  3986 consensus_meta.cc:287] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.572s	user 0.004s	sys 0.005s
I20260709 13:13:56.631858  4084 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:56.636811  3842 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.002s
I20260709 13:13:56.637382  3842 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.640210  4068 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:56.642464  3961 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.004s
I20260709 13:13:56.642544  3998 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.627287  3986 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.645753  3986 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:56.651111   848 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:13:56.643141  4045 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.685253   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.543s (SENT) [suppressed 34 similar messages]
I20260709 13:13:56.659862  3998 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.723300   926 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:13:56.725584  1567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.023s	sys 0.023s
W20260709 13:13:56.726046  1567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.023s	sys 0.023s
W20260709 13:13:56.726491  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 641615us
I20260709 13:13:56.728657   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:56.728765  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:56.712133  4094 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:56.734616   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.224293  4110 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:13:56.729888  3772 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:56.219312  3855 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.225764  3858 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.198866  3913 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:56.747781   847 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1705) took 2078 ms (client timeout 1651 ms). Trace:
I20260709 13:13:56.750941  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:56.718210  3842 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:56.762461  1593 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:13:56.763828   847 rpcz_store.cc:269] 0709 13:13:54.669334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.669470 (+   136us) service_pool.cc:224] Handling call
0709 13:13:56.746987 (+2077517us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b54003b10d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:56.747697 (+   710us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218466688}
I20260709 13:13:56.605073  3982 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.718437  3998 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.198843  4106 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.649s
I20260709 13:13:56.690007  4045 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.776696  4121 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:56.714283  4065 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:56.778203  3913 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:56.780146  3998 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:56.780308  3982 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:56.784055  1001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.012s	sys 0.060s
W20260709 13:13:56.813195  1001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.012s	sys 0.060s
W20260709 13:13:56.786245  1000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.002s	sys 0.011s
I20260709 13:13:56.794330   923 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:56.794703  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:56.818476  1000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.003s	sys 0.011s
I20260709 13:13:56.818925  1593 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:56.783735  3722 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.838802  1597 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:56.838991  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1763) took 1215 ms. Trace:
I20260709 13:13:56.839869  1001 rpcz_store.cc:276] 0709 13:13:55.623875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.048987 (+425112us) service_pool.cc:224] Handling call
0709 13:13:56.051173 (+  2186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.051416 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:56.084976 (+ 33560us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.838499 (+753523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.838589 (+    90us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:56.838653 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.838743 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.838913 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26660,"prepare.queue_time_us":47578,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":4217,"thread_start_us":711402,"threads_started":1}
I20260709 13:13:56.839591  1597 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:13:56.842308   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.842993   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.797724  4065 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.841681  3722 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.787719  4121 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:56.849301  3722 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:56.855540  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 75779.9us
I20260709 13:13:56.843282  1597 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:56.878054   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 192815us
I20260709 13:13:56.897989   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:56.900836  4106 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:56.901330  4106 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:56.901790  4106 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:56.903901  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:13:56.910292  3781 consensus_meta.cc:287] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.000s
W20260709 13:13:56.924244   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.970768   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.971872   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.976783   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:56.977622   848 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:13:56.979018   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 81105.3us
I20260709 13:13:56.998899  1566 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:57.047845   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.048516   851 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:57.049765   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.050765  4131 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:57.050904  4122 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:57.051496  3986 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:57.052078  3986 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:57.051704  4122 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.053097   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.052942  3986 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.053668   995 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 3.
I20260709 13:13:57.054656  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.055384  1003 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:13:57.089955  1597 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.096136  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:57.097295  1593 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
W20260709 13:13:57.139730   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.144918  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1775) took 1369 ms. Trace:
I20260709 13:13:57.145418  1000 rpcz_store.cc:276] 0709 13:13:55.775014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.135193 (+360179us) service_pool.cc:224] Handling call
0709 13:13:56.135545 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.158620 (+ 23075us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:56.272128 (+113508us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.999692 (+727564us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.144428 (+144736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.144509 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:57.144579 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.144669 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.144846 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170559,"prepare.queue_time_us":226195,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":79807,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":546558,"threads_started":1,"wal-append.queue_time_us":575751}
I20260709 13:13:57.182008   851 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
W20260709 13:13:57.188730   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.910951  3781 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.212553   851 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.224937   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.225961   851 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:57.229936  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.239939  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.240670  1001 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:57.054520  4122 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:57.231426  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:57.247205   848 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:57.144045  3982 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.265396   851 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:13:57.224227  4121 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.252113  3920 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.280345  1597 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
W20260709 13:13:57.278183   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 45838.4us
I20260709 13:13:57.232681  3961 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:57.281139  1597 pending_rounds.cc:85] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 6
I20260709 13:13:57.281713   848 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:57.264343  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:57.294411  1597 pending_rounds.cc:107] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:13:57.294783  1597 raft_consensus.cc:2889] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:57.294864  3961 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.295723  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:57.281983   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:57.292603   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.297019   853 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:57.297504  4131 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:57.259770  4068 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:13:57.314203  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.006s	sys 0.006s
W20260709 13:13:57.314579  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.006s	sys 0.006s
I20260709 13:13:57.297670  3961 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:57.330075   848 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:57.324198   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.299885  4121 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.266543  3982 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.311944  4131 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:57.321823  3803 consensus_meta.cc:287] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.207s	user 0.014s	sys 0.000s
I20260709 13:13:57.347718  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:57.316346  4123 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.299376  3920 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:57.353232   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 93178.3us
I20260709 13:13:57.349092  4131 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.356983  4111 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:57.357348  3961 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.357914  3961 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.360397  3961 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:57.359472   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.341447  3913 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.346933  4084 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:57.315968  4068 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.364245  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:57.377462  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 282651us
I20260709 13:13:57.354414  4123 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:57.367300  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 71675.1us
I20260709 13:13:57.367488   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.368287   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.341413  4128 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:57.365063  1592 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:57.384150   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.387159  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.387939  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.346767  4121 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.400440  4128 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.407954  4128 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.410418   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1956) took 2056 ms. Trace:
I20260709 13:13:57.414706  3955 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.415081  3920 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.415241  3955 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.410974   846 rpcz_store.cc:276] 0709 13:13:55.354086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.360646 (+  6560us) service_pool.cc:224] Handling call
0709 13:13:56.747111 (+1386465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.747302 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.747331 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.747374 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:56.747417 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.747478 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.410169 (+662691us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003b10d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:57.410336 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3187686144}
I20260709 13:13:57.381821  4126 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:57.375909  4137 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:57.351827  3803 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.416460   995 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:13:57.417194   997 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:13:57.417941  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.419203   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.442402   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1792) took 1407 ms. Trace:
I20260709 13:13:57.382135  3913 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.420466  4085 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:57.420364  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.401827  4123 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:57.428046  4129 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 13:13:57.459846   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.427861  4128 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.428658  3857 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:57.427495   895 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.660s (SENT)
W20260709 13:13:57.434082   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 44626.8us
I20260709 13:13:57.462859   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:57.432890  4131 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.443491  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:57.421118  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.443114   926 rpcz_store.cc:276] 0709 13:13:56.035308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.766500 (+731192us) service_pool.cc:224] Handling call
0709 13:13:56.788968 (+ 22468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.789250 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:57.260038 (+470788us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.442080 (+182042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.442133 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:57.442176 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.442242 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.442353 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":550592,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":626370,"threads_started":2}
W20260709 13:13:57.454061   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 111257us
I20260709 13:13:57.454829   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.455894   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.445669  4085 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:57.458211  4065 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.001s
I20260709 13:13:57.404704  4084 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:13:57.465454   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.434222  4126 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:57.434533  4137 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.478317   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.445430  4083 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:13:57.476138  3920 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:57.486222   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58773.4us
I20260709 13:13:57.478958  3781 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.488770  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.475714  4137 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.492213   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:57.494102   895 connection.cc:256] RPC call timeout handler was delayed by -1.4248789370001305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
W20260709 13:13:57.496789   895 connection.cc:256] RPC call timeout handler was delayed by -1.3348789369999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
I20260709 13:13:57.479933  4125 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.499643  3913 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.506243  4137 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.506623  3843 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:57.471840  3955 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:57.509279   923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 392) took 2198 ms (client timeout 1914 ms). Trace:
I20260709 13:13:57.509024  4137 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:57.509678   923 rpcz_store.cc:269] 0709 13:13:55.310790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.311770 (+   980us) service_pool.cc:224] Handling call
0709 13:13:57.509052 (+2197282us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400090454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:57.509179 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3382731008}
I20260709 13:13:57.510792  4068 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:57.510879   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.511910  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:57.512226   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1761) took 1893 ms. Trace:
I20260709 13:13:57.512566   999 rpcz_store.cc:276] 0709 13:13:55.619085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.002368 (+383283us) service_pool.cc:224] Handling call
0709 13:13:56.128611 (+126243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.129270 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.512052 (+1382782us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000ebd50. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:57.512174 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235456256}
I20260709 13:13:57.513000  4084 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.513685   921 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:57.515332   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:57.517297  4040 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:13:57.517904  4085 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.470451  3857 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:57.520134  4133 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:13:57.520417  4141 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:57.520795  4118 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 13:13:57.486100  3803 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.521026  3857 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.485714  1594 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:57.521611  3803 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.522279  4126 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.523485  4126 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.524048   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1790) took 1489 ms. Trace:
I20260709 13:13:57.523427  4125 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.525238   923 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: [6.6-7.7]
I20260709 13:13:57.524473   928 rpcz_store.cc:276] 0709 13:13:56.034920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.752803 (+717883us) service_pool.cc:224] Handling call
0709 13:13:56.753122 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:56.772337 (+ 19215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.772388 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:56.774791 (+  2403us) log.cc:844] Serialized 26 byte log entry
0709 13:13:57.240156 (+465365us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:57.441911 (+201755us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.451937 (+ 10026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.452016 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:57.523857 (+ 71841us) raft_consensus.cc:1685] finished
0709 13:13:57.523876 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.523994 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9687,"prepare.queue_time_us":122961,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":28510,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4660,"spinlock_wait_cycles":15771904,"thread_start_us":606416,"threads_started":2,"wal-append.queue_time_us":570739}
I20260709 13:13:57.526984  4125 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:57.530259  4106 consensus_meta.cc:287] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.000s
I20260709 13:13:57.530773  4106 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.541253  4045 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:57.542768  4106 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:57.543800  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:13:57.532932   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.536790  3955 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.545648   928 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:57.537073  4131 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.547791   997 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.548209   996 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:57.549373  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.538023  4121 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.539266   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.555828   999 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.558094  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.558691  1003 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.558804  4119 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:57.559247  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.560426  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.559258  4119 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:57.563227  4119 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.564101   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.564338  4119 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:57.539407  3982 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.539278  4128 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.566028   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.566603   923 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:13:57.567539   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:57.568444   851 raft_consensus.cc:2393] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:57.567559   996 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:57.591989  1003 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:13:57.610381   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 118251us
I20260709 13:13:57.614714  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:57.617748   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 150058us
I20260709 13:13:57.614390   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1794) took 1578 ms. Trace:
I20260709 13:13:57.618566   969 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.632020  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.632954  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.634181  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.634986   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1763) took 1647 ms. Trace:
I20260709 13:13:57.635329   925 rpcz_store.cc:276] 0709 13:13:55.988063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.988185 (+   122us) service_pool.cc:224] Handling call
0709 13:13:55.989377 (+  1192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.134061 (+144684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.634785 (+1500724us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000bc054. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:57.634933 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331122432}
I20260709 13:13:57.636404   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.637287  3956 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.637764   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.638576  3956 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.639566   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.640429  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1765) took 2012 ms. Trace:
I20260709 13:13:57.640712   997 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:57.640853  1004 rpcz_store.cc:276] 0709 13:13:55.628235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.050663 (+422428us) service_pool.cc:224] Handling call
0709 13:13:56.085583 (+ 34920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.553930 (+468347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.640168 (+1086238us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540034eed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:57.640366 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729682304}
I20260709 13:13:57.640902   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:57.642441  3956 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.643060  3956 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:57.643428  3956 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:57.643823  3956 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:57.644984  4087 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:57.647281   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.648267  4087 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.627349   930 rpcz_store.cc:276] 0709 13:13:56.035770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.516268 (+1480498us) service_pool.cc:224] Handling call
0709 13:13:57.516551 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.516749 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:57.540380 (+ 23631us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.613722 (+ 73342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.614124 (+   402us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:57.614166 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.614226 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.614325 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15967,"prepare.queue_time_us":23490,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":3553,"spinlock_wait_cycles":104064,"thread_start_us":72959,"threads_started":1,"wal-append.queue_time_us":68086}
I20260709 13:13:57.652573  3956 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.654711  3956 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:57.654911  4102 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.683s	user 0.006s	sys 0.021s
W20260709 13:13:57.665568  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131338us
W20260709 13:13:57.691394   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.694986   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.699553  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:57.704566   896 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:57.705694   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:57.710568  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:57.713289   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:57.666824  3982 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:57.718812   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.719465   997 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
W20260709 13:13:57.727900   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 230563us
W20260709 13:13:57.728629   895 connection.cc:256] RPC call timeout handler was delayed by -1.2430789370000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:13:57.729176   895 connection.cc:256] RPC call timeout handler was delayed by -1.2016789369999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 13:13:57.706694  3956 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:57.729738   895 connection.cc:256] RPC call timeout handler was delayed by -1.0945789370000512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 13:13:57.717052  4102 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.062s	user 0.004s	sys 0.000s
I20260709 13:13:57.731727   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.670397  3926 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:13:57.734201   997 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:13:57.735893   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.737233   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:57.725689  4013 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.738765   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.727087  3857 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:57.720074  3722 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.711939  4106 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:57.751291  1003 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:13:57.743147  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.700026  4073 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:13:57.718220  4076 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:13:57.745504  4070 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.741338  4013 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:57.760295  4045 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:57.760475  4013 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.757881  3857 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.738718  4034 raft_consensus.cc:476] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 13:13:57.757908   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.763731  3857 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.764642  4073 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:57.765079  4073 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.765942  4073 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.746039  3722 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.753409  4100 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:57.707527  4123 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:57.754946   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.732599  3956 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.774361   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.774469   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:57.775015   851 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.733254  3982 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:57.775246   848 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:57.756934  1001 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:57.757123   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:57.758761   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:57.769390  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.769248   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.780383   846 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.780519  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.781616  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:57.782529   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.770457   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:57.772249  4045 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.784220   996 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.771353   995 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:13:57.789657  1002 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:57.784760   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.780602  3982 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.774464  3956 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.798146  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:57.783758  4073 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.802366   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:57.803983   851 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:57.805604  4023 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:57.779388  3986 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:57.805665  3926 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.804816   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.806258  4023 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.794888  4021 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:57.812247   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:57.808468   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51394.7us
I20260709 13:13:57.822413  3920 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:57.824582  4150 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:57.825188  4150 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:57.826097  4150 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.816320  4126 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:57.828384  4126 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.828527   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1783) took 1795 ms. Trace:
I20260709 13:13:57.829010   921 rpcz_store.cc:276] 0709 13:13:56.033555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.749122 (+715567us) service_pool.cc:224] Handling call
0709 13:13:57.513474 (+764352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.514216 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.828282 (+314066us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b5400090450. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:57.828454 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756864768}
I20260709 13:13:57.812942  3986 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.828855  4126 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:57.816208  4023 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.848820   853 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:57.850116   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:57.807132  4119 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:57.852761   846 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:57.837214  4037 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.855511   969 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:57.858351  4144 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:57.859359  4145 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:57.860371  4100 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:57.859676  3955 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:57.817176  4021 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:57.880004  4021 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.864173   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1769) took 1864 ms. Trace:
I20260709 13:13:57.865665  3986 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.880786   924 rpcz_store.cc:276] 0709 13:13:55.999475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.003797 (+  4322us) service_pool.cc:224] Handling call
0709 13:13:57.863957 (+1860160us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b5400072e54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:57.864114 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144217984}
I20260709 13:13:57.855327  4106 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 13:13:57.888445  4060 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 2.969s	user 0.016s	sys 0.027s
I20260709 13:13:57.893106  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 567) took 3733 ms. Trace:
I20260709 13:13:57.896479  4068 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:57.896914  4068 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.893620  1596 rpcz_store.cc:276] 0709 13:13:54.159691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.159883 (+   192us) service_pool.cc:224] Handling call
0709 13:13:54.160184 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.534952 (+374768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.535004 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.539652 (+  4648us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.796197 (+256545us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:54.796237 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.320246 (+2524009us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400464bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:57.320301 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:57.887478 (+567177us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:57.892742 (+  5264us) raft_consensus.cc:1685] finished
0709 13:13:57.892777 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.892963 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10754,"raft.queue_time_us":92291,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":71,"spinlock_wait_cycles":352921216,"thread_start_us":288161,"threads_started":2,"wal-append.queue_time_us":223760}
I20260709 13:13:57.897748  4068 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.898950  4068 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.866555  4045 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.871801  3955 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:57.867513  3956 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.903144  3955 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.876156   996 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:13:57.876573  4123 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:13:57.877491  4148 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:13:57.904633  3956 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:57.907112  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.909214  3429 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.915755  3429 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.910295  4070 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:13:57.910470   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.921731  3717 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:57.922125  3717 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.922999  3717 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.923491  4137 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:57.924398  3717 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:57.878262  4151 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:13:57.858070  4119 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:57.926803  4119 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.930454   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:57.878485  4023 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:57.878755   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.935570   847 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.879446  3722 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:57.883287  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:57.937001  1003 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:57.883793  1004 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:57.934652  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:57.932029  4064 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.954546   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:13:57.976889  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.975243  3920 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:13:57.953558  3858 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:13:57.980079  3920 consensus_queue.cc:1243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:57.980296   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:57.980572  3920 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:57.980922   853 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.980958  3920 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:57.981928  3920 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:57.983008  3920 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:57.985144   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.978035  3722 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:57.985931  3722 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:57.986747   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:57.987211   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.987695   927 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:57.988766   846 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:57.990291   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.991343   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.991959   930 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 3.
I20260709 13:13:57.992206   921 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:57.992767   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.993134   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.993747   921 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:57.993993   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:57.994483   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:57.990332  1596 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:57.995898  1567 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:57.991348   847 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:58.005944   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:13:58.006083   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:58.009820  1002 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.011466  1000 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.015923   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.020304   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.021381   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.016726   925 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:57.987965  3429 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.031255   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.034615   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.005620  4056 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:58.022730  3986 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.012523  3858 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:58.060985   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.062734   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1949) took 2711 ms. Trace:
I20260709 13:13:58.064878   854 rpcz_store.cc:276] 0709 13:13:55.351062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.351234 (+   172us) service_pool.cc:224] Handling call
0709 13:13:55.351470 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.351693 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:55.579623 (+227930us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.848949 (+269326us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.423447 (+1574498us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18004059a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:13:57.654190 (+230743us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:58.062096 (+407906us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b18004059a0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cff9ccc0a 00007f0d0910c219 00007f0cffa54a40 00007f0cffa4f4e4 00007f0cffa4f455 00007f0d02b37060 00007f0d02b49716 00007f0d02b5a287 00007f0d02b5a208
0709 13:13:58.062270 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:58.062340 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:58.062402 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.062492 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.062660 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5249,"prepare.queue_time_us":261131,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1551,"raft.queue_time_us":2220378,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":93,"spinlock_wait_cycles":13491968,"thread_start_us":438769,"threads_started":2,"wal-append.queue_time_us":347338}
W20260709 13:13:58.067006   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.067040   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 2.19364e+06us
I20260709 13:13:58.067955   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:58.039171  4111 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:58.067943   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.039773  4056 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:58.076933  4085 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.077615  4107 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:58.080519   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.081887  3858 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.090276   921 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:13:58.084421  1595 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:13:58.084898   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.092011   819 connection.cc:256] RPC call timeout handler was delayed by -1.0958861650001381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:57.986791  3722 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.092689   853 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:13:58.096637   854 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:58.099651   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:13:58.101648   821 connection.cc:256] RPC call timeout handler was delayed by -2.0410851809999713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:13:58.101938   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:58.102169   821 connection.cc:256] RPC call timeout handler was delayed by -1.985285180999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:13:58.102638   821 connection.cc:256] RPC call timeout handler was delayed by -1.9807851809998738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:13:58.101943   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.103425  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:13:58.103111   821 connection.cc:256] RPC call timeout handler was delayed by -1.9114851809999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:13:58.106118   821 connection.cc:256] RPC call timeout handler was delayed by -1.8880851809999513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:13:58.106495   821 connection.cc:256] RPC call timeout handler was delayed by -1.8475851809998858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 13:13:58.106858   821 connection.cc:256] RPC call timeout handler was delayed by -1.7881851810000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
W20260709 13:13:58.107227   821 connection.cc:256] RPC call timeout handler was delayed by -1.7512851809999574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 13:13:58.107583   821 connection.cc:256] RPC call timeout handler was delayed by -1.689185180999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:13:58.107964   821 connection.cc:256] RPC call timeout handler was delayed by -1.6477851810000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260709 13:13:58.108352   821 connection.cc:256] RPC call timeout handler was delayed by -1.6135851809999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:13:58.108716   821 connection.cc:256] RPC call timeout handler was delayed by -1.5977490970001327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 13:13:58.109076   821 connection.cc:256] RPC call timeout handler was delayed by -1.5892851809999229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:13:58.109431   821 connection.cc:256] RPC call timeout handler was delayed by -1.5667851809998865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 13:13:58.109786   821 connection.cc:256] RPC call timeout handler was delayed by -1.5307851809998283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 13:13:58.110146   821 connection.cc:256] RPC call timeout handler was delayed by -1.4618490969999129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 13:13:58.110503   821 connection.cc:256] RPC call timeout handler was delayed by -1.4605851809999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 13:13:58.110873   821 connection.cc:256] RPC call timeout handler was delayed by -1.4560851809999349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 13:13:58.111239   821 connection.cc:256] RPC call timeout handler was delayed by -1.4551851809999334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 13:13:58.114619  3842 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.117273  3970 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:58.117883  3970 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.090354  3858 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:58.120527  3858 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.121395  3852 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:58.121742  3858 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.121907  3852 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.051585  4055 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:13:58.123744   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55906.1us
I20260709 13:13:58.125713   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:58.126842   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:58.127920   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.128113  4090 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:58.129030   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.124773  3855 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:58.129925  3855 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.130226   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.131004   928 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:58.127942  4032 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.132076  4032 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:58.132436  4032 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.133605   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.120896  4037 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:58.134246   821 connection.cc:256] RPC call timeout handler was delayed by -1.1480307629999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:13:58.134768   821 connection.cc:256] RPC call timeout handler was delayed by -1.0763871939998353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:13:58.135219   821 connection.cc:256] RPC call timeout handler was delayed by -1.050342813000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 13:13:58.135670   821 connection.cc:256] RPC call timeout handler was delayed by -1.0158576049998373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:13:58.088578  4056 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.142692   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.143672   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.144290   847 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:13:58.146595   854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:58.147135   854 raft_consensus.cc:606] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.147466   854 raft_consensus.cc:614] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:58.066038  3960 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:13:58.054471  3986 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.090999  4034 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:58.156369   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.151723   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.164997   846 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:58.124107  3832 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:58.181640   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:58.182139   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76501.3us
I20260709 13:13:58.182641   853 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:58.181568   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.140654  3772 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.124027  3717 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.128113  4127 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:58.103099  4111 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:58.180539   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.181857   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.184082   926 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.166445   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.195632   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.196466   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.196659   846 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:58.197680   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.198588   846 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:58.184312  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:58.191274  1001 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.191413   930 raft_consensus.cc:2410] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:13:58.203591   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.204474  4073 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:58.205168  4073 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:58.144246  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 107067us
I20260709 13:13:58.210471  4119 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:58.212426  4119 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.212857  4119 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:13:58.213478  4073 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.213725  4119 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.215714  4119 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:58.219586  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.097328  4013 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.148407  4140 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:58.225718   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.226076   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.159696  3842 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:58.228753   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.229494   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.236553   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.237984   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.239283   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.239830   847 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:58.240641   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.241808   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.232996   821 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.117892  4126 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.165195  4023 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.250396   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.188362  3717 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:58.243590   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.252427   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.233840   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.243856   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.181185  3832 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:58.140849  4037 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.192417  3803 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:13:58.188277  3772 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:58.248039   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.249564  1601 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.240713   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.302992   847 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:13:58.305662   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1969) took 2313 ms. Trace:
I20260709 13:13:58.255321   846 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:13:58.256040   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.180927  4034 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.283290   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.188544  4127 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:58.293448   995 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:58.243918  3855 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:58.308794   926 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:13:58.311291   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.250319  3970 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:58.321491   967 leader_election.cc:400] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9.
I20260709 13:13:58.321929   967 leader_election.cc:403] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:58.322786  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:58.317999   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.323709   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.318514   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.326602   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.326745   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:58.328019   922 raft_consensus.cc:2410] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:13:58.307544   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.329062   847 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.327256   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:13:58.292486  3832 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.254041  3717 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.239679  4140 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.333282  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:58.338853   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.358984   854 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:13:58.360622   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.317011  4090 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.306118   855 rpcz_store.cc:276] 0709 13:13:55.992672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.999125 (+  6453us) service_pool.cc:224] Handling call
0709 13:13:55.999405 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.999587 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:56.004234 (+  4647us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.305296 (+2301062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:58.305365 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:58.305423 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.305495 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.305606 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2001175,"prepare.queue_time_us":121,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":2001954,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":299414,"threads_started":2,"wal-append.queue_time_us":89089,"wal-append.run_cpu_time_us":1331,"wal-append.run_wall_time_us":1104047}
I20260709 13:13:58.363384   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:58.369263   855 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.334815  3970 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.377907   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:13:58.384002   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 100793us
I20260709 13:13:58.378665   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.379398   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:58.385087   854 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.385588   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.330637  4106 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9.
I20260709 13:13:58.360854  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:13:58.386790   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.326570  3855 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:58.387847   847 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:13:58.389773   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:13:58.392582   847 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:58.360611  4087 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:58.393759   847 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [8.5-10.6]
W20260709 13:13:58.374598  4147 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:58.383344  4094 raft_consensus.cc:2764] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:58.408313   853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
W20260709 13:13:58.407055  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 90741.6us
I20260709 13:13:58.408963   853 raft_consensus.cc:606] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.409389   853 raft_consensus.cc:614] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:58.413389   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.383925  4168 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.414170   927 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:13:58.375779  4122 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:13:58.331686  4111 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.411374  3970 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.391005  4145 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:13:58.443396   853 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:58.412503  4142 raft_consensus.cc:2764] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:58.383277  4150 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:13:58.416886  4106 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:58.441043  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:58.397516  4087 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:13:58.441221   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55700.7us
I20260709 13:13:58.381641  3842 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.456494   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:58.488672  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 47725.9us
I20260709 13:13:58.433080  3986 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:58.457651   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.437430  4177 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:58.489795  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.447434  4159 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.284s	user 0.000s	sys 0.006s
I20260709 13:13:58.444092  4128 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:58.455929  4150 consensus_queue.cc:1243] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:13:58.500190  4157 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 13:13:58.501863  4160 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:13:58.504875  3926 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.505285  3926 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:58.505640   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:58.506351   928 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 4: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:13:58.506075  3926 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:13:58.508270  4170 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:13:58.509018  4021 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.003s
I20260709 13:13:58.509423  4021 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:58.510587  4127 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:58.512154  4162 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:13:58.512674   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.513454   927 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:13:58.513916  4178 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.514566   927 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:58.516196  4161 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
W20260709 13:13:58.517719  4166 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:13:58.518355  4126 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:58.519822  1594 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.520972  4177 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:58.521957  4177 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:58.524784   996 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:58.525432   927 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:58.490994   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.498082   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:58.528225   851 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:58.528944   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.531123   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:58.507242   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:58.507287  3926 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.537338   924 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:58.484043  4087 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.522667  4037 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.570068  4140 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.498390  4142 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:58.567914  4034 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.533516  4085 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:58.576835   851 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:59.013275   851 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:13:59.014845   851 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:59.021426   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1801) took 1035 ms. Trace:
I20260709 13:13:59.022034   929 rpcz_store.cc:276] 0709 13:13:57.985620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.988225 (+  2605us) service_pool.cc:224] Handling call
0709 13:13:59.021132 (+1032907us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54003cafd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:59.021287 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722762112}
W20260709 13:13:58.969780   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 582984us
I20260709 13:13:59.004856  4094 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:59.006042  4068 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.036188  4068 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.037441  4090 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.037992  4090 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.038614  4068 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.044035  3852 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:59.045643  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 589) took 1112 ms. Trace:
I20260709 13:13:59.046159  1596 rpcz_store.cc:276] 0709 13:13:57.933646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.935710 (+  2064us) service_pool.cc:224] Handling call
0709 13:13:57.936082 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.995362 (+ 59280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.045310 (+1049948us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400464bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:59.045565 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124352896}
I20260709 13:13:59.049208  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1734) took 1055 ms. Trace:
I20260709 13:13:59.049746  1001 rpcz_store.cc:276] 0709 13:13:57.994145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.997693 (+  3548us) service_pool.cc:224] Handling call
0709 13:13:57.998016 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:58.224944 (+226928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.048891 (+823947us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54000479d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:59.049130 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443995904}
I20260709 13:13:59.050802   851 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:59.054963  3926 consensus_meta.cc:287] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.510s	user 0.005s	sys 0.007s
I20260709 13:13:59.058830   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1737) took 1057 ms. Trace:
W20260709 13:13:59.010694  4032 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.878s	user 0.007s	sys 0.013s
I20260709 13:13:59.061391  4032 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.059443   996 rpcz_store.cc:276] 0709 13:13:58.001048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.001192 (+   144us) service_pool.cc:224] Handling call
0709 13:13:59.058549 (+1057357us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54003e2454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:59.058707 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191428480}
W20260709 13:13:59.000905  4056 consensus_meta.cc:287] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.838s	user 0.007s	sys 0.012s
I20260709 13:13:59.065346  4056 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.066537  3772 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.072921   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:59.074020  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 585) took 1319 ms. Trace:
I20260709 13:13:59.074252   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:58.969911   927 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:13:58.971125  4013 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:58.604063  4021 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:58.971596   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.532996  4145 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:13:59.007939   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 476896us
I20260709 13:13:59.006271  4107 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:59.009955   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.067400  4056 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.029461   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:59.034206   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.076776  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.078606  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.074460  1600 rpcz_store.cc:276] 0709 13:13:57.754719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.755450 (+   731us) service_pool.cc:224] Handling call
0709 13:13:57.755794 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.793906 (+ 38112us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:57.970651 (+176745us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.438573 (+467922us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:58.438663 (+    90us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:58.438705 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.073796 (+635091us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b5400138dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:59.073848 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.073965 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55214,"prepare.queue_time_us":266927,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":13774,"raft.queue_time_us":110482,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7920,"spinlock_wait_cycles":86994816,"thread_start_us":614238,"threads_started":3,"wal-append.queue_time_us":423787}
W20260709 13:13:59.093730   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.972488  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 376) took 1252 ms. Trace:
I20260709 13:13:58.561304   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:59.111155   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 1993) took 1341 ms. Trace:
I20260709 13:13:59.110574  1004 rpcz_store.cc:276] 0709 13:13:57.720599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.720733 (+   134us) service_pool.cc:224] Handling call
0709 13:13:57.756692 (+ 35959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.952632 (+195940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.586056 (+633424us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400047ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:58.972418 (+386362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291872512}
I20260709 13:13:59.097749  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:59.099939   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:59.115592  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:13:59.111744   849 rpcz_store.cc:276] 0709 13:13:57.769539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.769660 (+   121us) service_pool.cc:224] Handling call
0709 13:13:58.561105 (+791445us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:13:59.111073 (+549968us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3882752}
W20260709 13:13:59.122112   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.063560  4032 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.157811   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1823) took 1036 ms. Trace:
I20260709 13:13:59.158418   924 rpcz_store.cc:276] 0709 13:13:58.121155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.128819 (+  7664us) service_pool.cc:224] Handling call
0709 13:13:59.157535 (+1028716us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54004457d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:59.157732 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937004800}
I20260709 13:13:59.124480   996 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:59.099637  3772 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.138931   854 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:59.095119  4171 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:13:59.144712   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.145637   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1996) took 1372 ms. Trace:
I20260709 13:13:59.165024   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:59.168493   848 rpcz_store.cc:276] 0709 13:13:57.773507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.793597 (+ 20090us) service_pool.cc:224] Handling call
0709 13:13:57.809252 (+ 15655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.119797 (+310545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.119827 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.120328 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.341064 (+220736us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:58.341126 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.341219 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.145403 (+804184us) raft_consensus.cc:1685] finished
0709 13:13:59.145425 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.145573 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310246,"raft.queue_time_us":314420,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":220321,"threads_started":1,"wal-append.queue_time_us":1023397}
I20260709 13:13:59.147143   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:59.171552   847 raft_consensus.cc:606] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.172092   847 raft_consensus.cc:614] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:59.174597  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59157.9us
W20260709 13:13:59.152586   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 123217us
I20260709 13:13:59.177990   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:59.154783   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.180474   849 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:59.180397   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:59.180918  3843 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:59.181363  3843 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:59.181886   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1810) took 1191 ms. Trace:
I20260709 13:13:59.153702  4168 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.182482  4011 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:59.182297   921 rpcz_store.cc:276] 0709 13:13:57.990780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.994268 (+  3488us) service_pool.cc:224] Handling call
0709 13:13:59.181689 (+1187421us) spinlock_profiling.cc:243] Waited 95.2 ms on lock 0x7b5400174b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:59.181804 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218887040}
I20260709 13:13:59.183208  4011 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:59.184134  4011 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.186437  4128 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:59.186975  4128 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.187496  3852 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.189179   849 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:59.189735   849 raft_consensus.cc:606] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.190034   849 raft_consensus.cc:614] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:59.178515  3832 consensus_meta.cc:287] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.805s	user 0.018s	sys 0.001s
I20260709 13:13:59.094218  4087 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.154757  4056 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:59.176101   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1739) took 1173 ms. Trace:
I20260709 13:13:59.175889  4171 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.194433   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.098603  4090 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.147565  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:59.201467   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46793.3us
I20260709 13:13:59.198222   999 rpcz_store.cc:276] 0709 13:13:58.003130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.003263 (+   133us) service_pool.cc:224] Handling call
0709 13:13:58.003657 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.026224 (+ 22567us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:58.044734 (+ 18510us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.307755 (+263021us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.328565 (+ 20810us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.328907 (+   342us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:58.328971 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.329097 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.135331 (+806234us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b240023f3a8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff8d89c4 00007f0cff8d6acb 00007f0cffa91dc9 00007f0cffa91a55 00007f0cffa90fb0 00007f0d02707488 00007f0d02707be8 00007f0d026df398 00007f0d026da76a 00007f0d026d9bb6
0709 13:13:59.167886 (+ 32555us) raft_consensus.cc:1685] finished
0709 13:13:59.175895 (+  8009us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.176034 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22721,"prepare.queue_time_us":24616,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":27677,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":17950,"spinlock_wait_cycles":42864896,"thread_start_us":216148,"threads_started":1,"wal-append.queue_time_us":333821}
W20260709 13:13:59.203552  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56096.5us
I20260709 13:13:59.206696   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.177651  4068 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.192586  4189 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:59.196080  3852 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:59.238898   849 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:59.239032   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.177799  3926 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:13:59.260794   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.220921  3803 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.262089   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:59.235769  4169 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:59.219156  4184 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:59.243175  4168 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.200861  3832 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.240347  4189 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.182318  3843 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:59.349550   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 65037.3us
I20260709 13:13:59.234275  4066 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:13:59.208194  4056 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.363826  4189 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.257138  4068 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.261195  4180 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:13:59.342634  4032 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.266816  4188 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:59.342612  4021 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.278394  4190 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.374262   851 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:59.388092   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:59.349777  4168 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:59.359305  4169 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:13:59.397938   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 229680us
W20260709 13:13:59.398065   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 220952us
W20260709 13:13:59.399588   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 137541us
I20260709 13:13:59.373539  4094 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.367252  3803 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:13:59.402369   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:13:59.403515   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.408689   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:13:59.396142  4168 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.401139  3843 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.426368  4068 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.414155  4032 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.427384  4090 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.410617  4188 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.429289  4087 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:59.392318  4021 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.432968  4169 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.433815  4021 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.436335   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1719) took 1277 ms. Trace:
I20260709 13:13:59.436779  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.437505  1000 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:13:59.436725   855 rpcz_store.cc:276] 0709 13:13:58.158592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.393241 (+234649us) service_pool.cc:224] Handling call
0709 13:13:58.393689 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.972309 (+578620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.972339 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.005513 (+ 33174us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.210306 (+204793us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:59.240024 (+ 29718us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.257101 (+ 17077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.257183 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.436091 (+178908us) raft_consensus.cc:1685] finished
0709 13:13:59.436112 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.436273 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10502,"prepare.queue_time_us":32645,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":59652,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":10597,"thread_start_us":676331,"threads_started":2,"wal-append.queue_time_us":391752}
I20260709 13:13:59.438894   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:59.440896  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.441515  1000 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:59.441550   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.442946  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:13:59.397540  4121 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:13:59.452991   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:59.453680   855 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:13:59.456539   999 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:59.412794  4171 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:59.436223  3960 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.466583   853 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:59.448203  4021 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.458353  3852 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:13:59.452173  4188 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:59.471010  3852 consensus_queue.cc:1243] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:59.474812   998 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.476537  3926 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:59.478312  4164 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 1.080s	user 0.009s	sys 0.003s
I20260709 13:13:59.473870  3832 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:59.490401   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 91196.1us
I20260709 13:13:59.481128  4021 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.485992  4188 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:59.480155  3960 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.492949  4202 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:59.493701  4021 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.517230   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.517890   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.505743  4203 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.522658  4203 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:59.526108  4194 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:13:59.528030  4195 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:59.528687   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 1973) took 3534 ms. Trace:
I20260709 13:13:59.529062   852 rpcz_store.cc:276] 0709 13:13:55.993918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.007631 (+ 13713us) service_pool.cc:224] Handling call
0709 13:13:56.007909 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.082676 (+2074767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.082703 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.083161 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.042431 (+959270us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b1800405940. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:13:59.233208 (+190777us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:59.233249 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.233662 (+   413us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.528389 (+294727us) raft_consensus.cc:1685] finished
0709 13:13:59.528410 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.528625 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2074281,"raft.queue_time_us":2074958,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"spinlock_wait_cycles":4629888,"thread_start_us":190545,"threads_started":1,"wal-append.queue_time_us":1318429}
W20260709 13:13:59.534104   922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1800) took 1548 ms (client timeout 1696 ms). Trace:
W20260709 13:13:59.534550   922 rpcz_store.cc:269] 0709 13:13:57.985332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.986527 (+  1195us) service_pool.cc:224] Handling call
0709 13:13:59.533907 (+1547380us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400445cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:13:59.534042 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782802176}
I20260709 13:13:59.537241  4205 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:13:59.537859  4205 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.540635   852 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:59.544615   846 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:59.551996  4056 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:59.505211  4168 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.552888  4188 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.555223  4168 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.560240   846 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:59.562543   846 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:59.515395  4202 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:59.571630  4121 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.587407  4056 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.588317  4056 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.539511   749 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 4 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:59.595862  4056 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.572243  4121 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.565142  4127 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.593573   749 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 4 to 5, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 5 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.605690   749 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.597782  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.608660   749 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 4 to 5, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:59.611622   749 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.584146  4203 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:59.615258   753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 226) took 16125 ms (client timeout 15000 ms). Trace:
I20260709 13:13:59.614421   749 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 6 to 7, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 7 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.626484   749 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:59.633343   749 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.634756  3960 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:59.617076   753 rpcz_store.cc:269] 0709 13:13:43.490012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.490187 (+   175us) service_pool.cc:224] Handling call
0709 13:13:59.615183 (+16124996us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:52.834926 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:52.835585 (+   659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:52.835665 (+    80us) write_op.cc:435] Acquired schema lock
0709 13:13:52.835678 (+    13us) tablet.cc:662] Decoding operations
0709 13:13:52.837111 (+  1433us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:52.837183 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:52.837214 (+    31us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:13:52.838550 (+  1336us) tablet.cc:701] Row locks acquired
0709 13:13:52.838595 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:13:52.838773 (+   178us) write_op.cc:270] Start()
0709 13:13:52.838911 (+   138us) write_op.cc:276] Timestamp: P: 1783602832838709 usec, L: 0
0709 13:13:52.838928 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:13:52.861788 (+ 22860us) log.cc:844] Serialized 6660 byte log entry
0709 13:13:54.092535 (+1230747us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:54.595562 (+503027us) op_driver.cc:464] REPLICATION: finished
0709 13:13:54.792390 (+196828us) write_op.cc:301] APPLY: starting
0709 13:13:54.824279 (+ 31889us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:54.929610 (+105331us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:54.943672 (+ 14062us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:54.945743 (+  2071us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:54.946147 (+   404us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:13:54.946205 (+    58us) write_op.cc:312] APPLY: finished
0709 13:13:54.946686 (+   481us) log.cc:844] Serialized 119 byte log entry
0709 13:13:54.949917 (+  3231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:54.950098 (+   181us) write_op.cc:454] Released schema lock
0709 13:13:54.950172 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:55.014256 (+ 64084us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:55.015533 (+  1277us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46544,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":196478,"apply.run_cpu_time_us":5059,"apply.run_wall_time_us":210807,"num_ops":17,"prepare.queue_time_us":225136,"prepare.run_cpu_time_us":52584,"prepare.run_wall_time_us":172840,"raft.queue_time_us":278312,"raft.run_cpu_time_us":26682,"raft.run_wall_time_us":218708,"replication_time_us":1756378,"spinlock_wait_cycles":52459776,"thread_start_us":489031,"threads_started":4,"wal-append.queue_time_us":181535,"wal-append.run_cpu_time_us":78875,"wal-append.run_wall_time_us":2908685}]]}
I20260709 13:13:59.644933   846 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:59.654314   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:59.654513   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.654981   930 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:13:59.658713   929 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:59.662423   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:13:59.673677   924 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:59.674000   923 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:13:59.674388   923 pending_rounds.cc:85] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 4
I20260709 13:13:59.674598   923 pending_rounds.cc:107] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:13:59.674813   923 raft_consensus.cc:2889] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:59.681401  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.682751  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.683629  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.683797   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:59.684726  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.682199  1000 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:13:59.689101   930 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:13:59.689553   930 pending_rounds.cc:85] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 4
I20260709 13:13:59.689666  1004 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:13:59.689810   930 pending_rounds.cc:107] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:13:59.690122   930 raft_consensus.cc:2889] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:59.644349  4178 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:59.692946   998 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:13:59.694347   928 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:59.694834   928 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:59.625463  4128 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.705206   848 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:59.673796  3960 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:13:59.697688  4177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.148s	user 0.036s	sys 0.046s
I20260709 13:13:59.693174  4045 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.709882   928 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.714448   928 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.712380   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:13:59.711395  4177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.036s	sys 0.047s
I20260709 13:13:59.718098   848 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:59.721619   995 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:13:59.723505   995 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:59.702284  4206 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.730480  4121 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.735661  4128 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.717499  3960 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.736117  4206 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.726786  4189 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.738276  3960 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:59.738534  4206 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:13:59.720047  4207 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:13:59.743891   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.740103  4189 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:59.750543   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.760105   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.713424  4045 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:13:59.701613   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.781810   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1728) took 1556 ms. Trace:
I20260709 13:13:59.784830   853 rpcz_store.cc:276] 0709 13:13:58.224883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.490917 (+266034us) service_pool.cc:224] Handling call
0709 13:13:58.491212 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.467050 (+975838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.781486 (+314436us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b54000c94d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:13:59.781712 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242290944}
I20260709 13:13:59.778527  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.788592   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:13:59.793927   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.772207  4211 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:13:59.821882  4021 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.822839   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:59.823089  4180 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:13:59.800204   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.825243   895 leader_election.cc:336] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.584s (SENT)
I20260709 13:13:59.827082   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:13:59.829110   895 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.651s (SENT)
I20260709 13:13:59.829542   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:59.775949  4045 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:13:59.804256   749 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 2 to 3, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 3 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:13:59.799546   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:59.799048  4171 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:59.799710  4177 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.796572  4213 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:59.818473  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 219688us
I20260709 13:13:59.829465   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:13:59.830467   853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:13:59.809561   928 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:59.836701   853 raft_consensus.cc:606] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.836450  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.837920   853 raft_consensus.cc:614] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:59.839340   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 379505us
I20260709 13:13:59.839568   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.841642   896 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:59.843152  4045 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.840531  4180 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:59.845721 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:59.847431 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:59.848232   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:13:59.837325  4057 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.847276   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.828251  4073 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:13:59.836958  4034 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.840919  4177 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:59.850013  4045 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.859858  4177 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:13:59.860695  4177 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.861169  4202 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.862345  4202 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.863631   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:59.854856  4021 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:13:59.869280   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.876099  4073 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:59.876569  4073 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:59.877461  4073 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.878768  4073 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.879042  4068 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:59.879448  4068 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:59.880333  4068 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.881404  4068 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.883085   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1733) took 1889 ms. Trace:
I20260709 13:13:59.883497   997 rpcz_store.cc:276] 0709 13:13:57.993885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.008031 (+ 14146us) service_pool.cc:224] Handling call
0709 13:13:58.008368 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:58.036859 (+ 28491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.036891 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:58.585653 (+548762us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4800210cd8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:13:58.587772 (+  2119us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b180027f640. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa202a6 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a
0709 13:13:59.548852 (+961080us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b2c000e6738. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:13:59.549423 (+   571us) log.cc:844] Serialized 52 byte log entry
0709 13:13:59.798254 (+248831us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:59.798293 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.798364 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.882838 (+ 84474us) raft_consensus.cc:1685] finished
0709 13:13:59.882857 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.883007 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20640,"raft.queue_time_us":20124,"raft.run_cpu_time_us":16938,"raft.run_wall_time_us":994823,"spinlock_wait_cycles":27424768,"thread_start_us":248468,"threads_started":1,"wal-append.queue_time_us":275434}
I20260709 13:13:59.891506  4209 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:59.893584  4209 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:59.853255   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:59.893967  4209 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:59.894718   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.837786  4171 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.892001  4094 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:59.859910  4045 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:59.850602  4057 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.896669  4045 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.896867  4057 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.898712  4171 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:13:59.848435  4188 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.899919  4064 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:59.900341  4064 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:13:59.900434   922 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:59.900754  4064 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.849826  4206 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.901477  4206 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:13:59.902299  4205 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:59.902848  4205 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:59.851984  4168 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:59.903429  4205 consensus_queue.cc:1243] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:59.906420  3955 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:13:59.893726   927 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:13:59.908885  4182 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.904s	user 0.011s	sys 0.001s
I20260709 13:13:59.906455  4126 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:13:59.918846   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 385) took 3874 ms. Trace:
I20260709 13:13:59.920192  4168 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:59.920594  4168 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:13:59.921154  4011 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:59.921486  4168 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.922806  4011 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:59.912542   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:13:59.851984  3960 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:13:59.924289  4123 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:59.924683  4123 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:59.924772  3843 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:59.925292  3843 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:13:59.925680  3843 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.925523  4123 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.904256   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:13:59.926882  4123 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.927764  3960 consensus_queue.cc:1243] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:59.928242  4127 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:59.903532  4206 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:13:59.928320  3960 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:59.922685  4094 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:59.929838   924 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:59.944335  4107 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.944447  4094 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:59.945021  4094 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:13:59.924865   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:13:59.945744   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:13:59.919345   850 rpcz_store.cc:276] 0709 13:13:56.044603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.046252 (+  1649us) service_pool.cc:224] Handling call
0709 13:13:56.046617 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:58.308252 (+2261635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.308289 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:58.333047 (+ 24758us) log.cc:844] Serialized 52 byte log entry
0709 13:13:58.434707 (+101660us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:58.830524 (+395817us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.831973 (+  1449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.832097 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.908195 (+1076098us) spinlock_profiling.cc:243] Waited 6.99 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:13:59.918585 (+ 10390us) raft_consensus.cc:1685] finished
0709 13:13:59.918683 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.918792 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2035723,"prepare.queue_time_us":695532,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":2037699,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":17113,"spinlock_wait_cycles":16081664,"thread_start_us":636950,"threads_started":3,"wal-append.queue_time_us":209776}
I20260709 13:13:59.947481   924 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:13:59.932226  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 84384us
I20260709 13:13:59.938339   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.940459  3960 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:13:59.940367  4206 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:59.912420  3717 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:59.922742  4168 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.946360   848 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:13:59.952152   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:00.000119   921 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
W20260709 13:13:59.931694   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 84474us
I20260709 13:13:59.952867   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.938916   926 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:13:59.948324   924 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:13:59.955950   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:13:59.956403   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 84774.5us
I20260709 13:13:59.957978   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:13:59.958007   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:13:59.960629   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.003500   924 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.003492   929 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:14:00.004515   929 pending_rounds.cc:85] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 3
I20260709 13:14:00.004894   929 pending_rounds.cc:107] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:14:00.005326   929 raft_consensus.cc:2889] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:59.999091  4127 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:14:00.007009   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:13:59.965725  3955 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.949992  3852 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:14:00.007773  3955 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:00.023988   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:00.026114   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 101317us
I20260709 13:14:00.017367   969 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:00.029093  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 77886.9us
I20260709 13:14:00.003932  4171 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.950145  4145 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 13:14:00.037954   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.042450  1002 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:14:00.047801  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 93164.4us
I20260709 13:14:00.050709   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:00.051491   995 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.027290  3852 consensus_queue.cc:1243] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:00.041531  4212 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:00.028280  4127 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:00.091645  1600 pending_rounds.cc:85] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 7
I20260709 13:14:00.094426  1600 pending_rounds.cc:107] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:14:00.094789  1600 raft_consensus.cc:2889] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.092692  1596 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:00.108542   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.109457  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:00.037886  4171 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.119187   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.894769  4209 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.052402  4090 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:00.056079  4083 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:00.139631   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 115498us
I20260709 13:14:00.127729  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:00.134656  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 102729us
I20260709 13:14:00.143129  1596 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:00.144078   930 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:00.145334   995 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:00.028110  4057 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:00.139338   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.145208   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:00.150156   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.141769   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:13:59.956051  4094 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.151268   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:00.151839  4094 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:00.150018  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:00.150876   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:00.122916  3722 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.156195   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:00.156030  3722 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:00.158294  3722 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:00.160605  4225 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:14:00.163444  4066 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:00.163968  4066 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:00.164878  4066 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.094138  3803 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:00.166157  4066 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:00.166261  3803 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.055017  4212 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.140436  4083 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:00.167841   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.168118  4083 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:00.093878  4127 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:00.135033   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81781.7us
I20260709 13:14:00.170433  4057 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:00.170744   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:00.170682  4127 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:00.147140  3960 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.175915  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
W20260709 13:14:00.188966   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47261.4us
I20260709 13:14:00.189970   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.191285   821 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.191948  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:00.196988  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:00.197772   924 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:14:00.198896   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.199148  1001 pending_rounds.cc:85] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 2
I20260709 13:14:00.199548  1001 pending_rounds.cc:107] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:14:00.199926  1001 raft_consensus.cc:2889] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.124785  4171 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.222488   922 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:00.203827   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.207674   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.224493   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.281016   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:00.170480  3852 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:00.192006  4209 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:00.281905   995 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:00.319638   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 92992.2us
W20260709 13:14:00.319698  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 138255us
I20260709 13:14:00.230722  4107 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.282569  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 139158us
I20260709 13:14:00.346873  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:00.230489  3956 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:00.330632  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:00.335743  1596 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:00.231901  4056 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:14:00.279194  3803 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:00.222568  4127 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.230389  4145 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.112s
W20260709 13:14:00.321256   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 93894.8us
I20260709 13:14:00.318928  3987 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:00.279152  4057 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.299851   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.338783  4200 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.358511  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:00.281847  3955 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:00.201862  4090 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:00.372406  4177 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.511s	user 0.021s	sys 0.003s
I20260709 13:14:00.366742  1601 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:00.372702   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.320670  3852 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.391568   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 185857us
I20260709 13:14:00.392215   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:00.421294   849 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:00.398092   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51847.1us
I20260709 13:14:00.398851   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:00.400153  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:00.327440  4073 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.259824  4085 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:14:00.349592  4013 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:00.329995  3982 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:14:00.419312   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.370960  4180 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.371338  4188 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:00.370640  4100 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:14:00.423913  3852 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:00.475013  4180 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:00.480460  4222 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:14:00.480641  4180 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:00.461836   849 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:00.482547   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:00.482610  1598 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
W20260709 13:14:00.408877  4170 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:14:00.455922  4219 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:00.365945  4045 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.365283  4107 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:00.485424   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.487726   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 133674us
I20260709 13:14:00.578047   894 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:00.499529   995 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:14:00.523958   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.580169   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.560475   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.564234   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:00.581141   849 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:00.582512   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:00.423060   927 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
W20260709 13:14:00.567672   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 148442us
W20260709 13:14:00.564507  4064 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.663s	user 0.016s	sys 0.013s
I20260709 13:14:00.584184   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.403028  4057 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:00.585613   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.584751  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:00.595202  3843 consensus_meta.cc:287] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.669s	user 0.018s	sys 0.007s
I20260709 13:14:00.424695  4177 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.601387   923 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:00.582657  4123 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.030s
W20260709 13:14:00.585832  4222 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
W20260709 13:14:00.584628  4228 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:14:00.411260  4200 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.463044  4073 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:00.609198   852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.007s
W20260709 13:14:00.610926   850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.016s
W20260709 13:14:00.596504  4202 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.006s
I20260709 13:14:00.610353   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:00.600468  4227 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 13:14:00.623323   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 276292us
W20260709 13:14:00.605304  4068 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.720s	user 0.005s	sys 0.022s
W20260709 13:14:00.626255   850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.005s	sys 0.016s
I20260709 13:14:00.604151  4242 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:14:00.628382   852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.008s
I20260709 13:14:00.601812  4219 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:14:00.633709  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49032.5us
W20260709 13:14:00.636255   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.685460   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.668995  4073 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:00.668898  4201 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:00.698851  4250 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.643992   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 64773.6us
I20260709 13:14:00.701395   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:00.604369  4212 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:00.681756   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2019) took 1246 ms. Trace:
I20260709 13:14:00.624754  4013 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:00.599320  3843 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:00.700812   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:00.673281  4213 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:00.596395  4064 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.710131   855 rpcz_store.cc:276] 0709 13:13:59.435329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.454225 (+ 18896us) service_pool.cc:224] Handling call
0709 13:13:59.454522 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:59.613067 (+158545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.613112 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:59.620831 (+  7719us) log.cc:844] Serialized 52 byte log entry
0709 13:14:00.095515 (+474684us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:00.123325 (+ 27810us) log.cc:844] Serialized 14 byte log entry
0709 13:14:00.123725 (+   400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.123890 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.170242 (+ 46352us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b18002ffaa0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa52522 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:00.629749 (+459507us) raft_consensus.cc:1685] finished
0709 13:14:00.681407 (+ 51658us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.681663 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19833,"prepare.queue_time_us":27589,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":29233,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":12696,"spinlock_wait_cycles":65798400,"thread_start_us":594034,"threads_started":2,"wal-append.queue_time_us":548312}
I20260709 13:14:00.699977  4224 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:14:00.719120   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.729424   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.729871   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.706638   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 117263us
W20260709 13:14:00.722373   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 138265us
I20260709 13:14:00.727269   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:00.703456  4202 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:00.768206  1003 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
W20260709 13:14:00.769981   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 47559.9us
I20260709 13:14:00.778769   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:00.708421  4068 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:00.713029  4013 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.720197  4229 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 13:14:00.786336   922 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:00.721704  4212 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:00.795221  4212 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:00.795739  4212 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:00.796375   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:00.789048   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:00.755810  4123 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.756314  3803 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:00.717227  3982 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:14:00.729067  4224 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:00.785863  4252 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:00.793588  4013 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:00.805339  4239 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:14:00.815969  4068 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:00.817479  4251 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:14:00.823930  4068 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.824918  4190 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:00.825273  4107 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:00.826838  4248 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:14:00.832508  4057 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:00.785610  3955 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:14:00.705637  4233 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:14:00.810683  3987 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.817953  4168 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.000s
I20260709 13:14:00.844215   995 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:00.787312  4100 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:14:00.800000  3852 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:00.853186  4232 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:14:00.814812  4123 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.833523  4073 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:00.856326  4045 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:00.854534  3987 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.982239 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:00.997915   846 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:00.998093   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:01.039878  3722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.011s	sys 0.014s
W20260709 13:14:01.040558  3722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.011s	sys 0.014s
I20260709 13:14:01.040580   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.001132   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.043112   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.043224   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:01.004082   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 174842us
W20260709 13:14:01.005748   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 395310us
I20260709 13:14:01.044198   848 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:14:01.045578   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.046494   848 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:14:01.007582   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2034) took 1068 ms. Trace:
I20260709 13:14:01.017670   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2040) took 1060 ms. Trace:
I20260709 13:14:00.884452  4177 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:01.021265  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 431095us
W20260709 13:14:01.021793  3960 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.011s	sys 0.022s
I20260709 13:14:01.030433  3852 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.002460   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2038) took 1062 ms. Trace:
I20260709 13:14:00.909823  4200 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:00.945559  4209 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.001s
I20260709 13:14:00.726094  4213 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:00.882545  4224 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.102077  4244 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.102304  4224 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:00.812391  4219 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.106688  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:00.937426  4238 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:14:01.110242  4263 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.699467  4250 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.114665  3960 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:01.070251  4241 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:01.073021  4064 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.122067  4250 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.122407  4241 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.075505   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.124888   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:01.125653  4241 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.115207  4263 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.130563  4263 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.080449   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.096335  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.106940   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:01.006266   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.132200   848 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:01.132114  1001 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:01.133318   926 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:01.135257   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.135504  4233 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:01.136246   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.136198  4233 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.137143  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.137917  1001 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:01.139322   853 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:01.139756   998 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:01.140702   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.142316   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.143726   853 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:01.133391   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.049093   854 rpcz_store.cc:276] 0709 13:13:59.938733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.939391 (+   658us) service_pool.cc:224] Handling call
0709 13:13:59.941819 (+  2428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.957083 (+ 15264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.957116 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.999952 (+ 42836us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.348841 (+348889us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:00.357628 (+  8787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.357755 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.006920 (+649165us) raft_consensus.cc:1685] finished
0709 13:14:01.006970 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.007364 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30169,"raft.queue_time_us":23777,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9267,"thread_start_us":294672,"threads_started":1,"wal-append.queue_time_us":427495}
I20260709 13:14:01.157382   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:01.132059   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.157691   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.161329   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.139207  4233 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.158156   854 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:01.162707  4253 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.167990   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.168292  4264 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:01.168816  4264 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.169282  4264 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.170154   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.162774   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.173566   848 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:01.179246  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:01.096378  4123 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:01.173702   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51608.3us
I20260709 13:14:01.182253   998 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:01.049338   852 rpcz_store.cc:276] 0709 13:13:59.957305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.981247 (+ 23942us) service_pool.cc:224] Handling call
0709 13:14:00.017415 (+ 36168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.023377 (+  5962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.023423 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.048144 (+ 24721us) log.cc:844] Serialized 494 byte log entry
0709 13:14:00.677798 (+629654us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:00.677859 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.677961 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.003341 (+325380us) raft_consensus.cc:1685] finished
0709 13:14:01.003387 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.017418 (+ 14031us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":629304,"threads_started":1,"wal-append.queue_time_us":887509}
I20260709 13:14:01.191502   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:01.205435   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.103068  4244 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.217741  4245 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.194221  4057 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.196806   855 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:01.163569  4253 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.228591   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.218703  4245 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.218791  4057 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.239943   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.241537   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.241912  4057 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.242383  3987 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.242974   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.243907   855 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:14:01.244612  4245 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.245360   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.249539  4213 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.247947   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.252405  4253 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.252847  4255 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:01.254232   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:01.262843  4188 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.264308   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.086527   851 rpcz_store.cc:276] 0709 13:13:59.939791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.939887 (+    96us) service_pool.cc:224] Handling call
0709 13:13:59.942152 (+  2265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.949802 (+  7650us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:00.027675 (+ 77873us) log.cc:844] Serialized 14 byte log entry
0709 13:14:00.631886 (+604211us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.632696 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.632869 (+   173us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:00.632918 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.633023 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.001873 (+368850us) raft_consensus.cc:1685] finished
0709 13:14:01.001913 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.002259 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66504,"prepare.queue_time_us":79307,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":391328,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":49809,"spinlock_wait_cycles":1186048,"thread_start_us":603701,"threads_started":2,"wal-append.queue_time_us":343809}
W20260709 13:14:01.269802   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78464.3us
I20260709 13:14:01.270751  4056 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:01.282812   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.286033  4244 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.286847   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.287402   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:01.292404   849 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:01.300239  3843 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.303997  4145 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:01.284775  4257 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.470s	user 0.000s	sys 0.004s
W20260709 13:14:01.307669  4266 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.475s	user 0.001s	sys 0.003s
I20260709 13:14:01.306471  4123 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Committing config change with OpId 8.7: config changed from index 3 to 7, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:01.313596  4212 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.011s
I20260709 13:14:01.314522  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 1762) took 1119 ms. Trace:
W20260709 13:14:01.317785  4066 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.151s	user 0.009s	sys 0.011s
I20260709 13:14:01.335515   855 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 4: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:01.327108  4083 consensus_meta.cc:287] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.158s	user 0.008s	sys 0.014s
I20260709 13:14:01.344290  1003 rpcz_store.cc:276] 0709 13:14:00.195207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.196608 (+  1401us) service_pool.cc:224] Handling call
0709 13:14:01.314040 (+1117432us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54000884d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:01.314306 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311480576}
I20260709 13:14:01.330428  4056 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:01.351699   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 97719.2us
I20260709 13:14:01.364750   854 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:01.367767  4258 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:01.370743   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.370813  4145 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.371735   996 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:01.372773  4057 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:01.373445  4145 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.353679  4212 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:01.379818  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 254224us
I20260709 13:14:01.386013   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.358139  4056 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.391037   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.392028   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1873) took 1447 ms. Trace:
I20260709 13:14:01.393098   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.394024   997 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:01.395888  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.396612  1002 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.399964  4057 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.400863  4057 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:01.402841   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 115586us
I20260709 13:14:01.403570  4057 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.403772   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:01.405184  4057 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:01.361693  4083 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.407445  4142 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:01.407948  4142 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.408085  1000 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:01.408380  4142 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:01.392608   923 rpcz_store.cc:276] 0709 13:13:59.937372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.937733 (+   361us) service_pool.cc:224] Handling call
0709 13:13:59.938005 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.602012 (+664007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.375896 (+773884us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400446bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:01.384498 (+  8602us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525053312}
I20260709 13:14:01.406265  4057 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:01.410804  4057 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:01.351306  4188 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.414763  4188 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:01.416755   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 61897.6us
I20260709 13:14:01.405566   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:01.405995   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2039) took 1465 ms. Trace:
I20260709 13:14:01.425770   853 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:01.405553  4212 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.428727  1002 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:01.430809   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.429487  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.434518   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.360853  4255 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.442134   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.448431   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.451849  4045 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:01.452498  4045 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.452996  4045 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:01.461987  4244 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:01.463886  4213 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:01.440515  4250 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:01.465247   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.465067  4250 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.418970   850 rpcz_store.cc:276] 0709 13:13:59.940147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.946915 (+  6768us) service_pool.cc:224] Handling call
0709 13:13:59.947287 (+   372us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:00.001201 (+ 53914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.001233 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:00.025218 (+ 23985us) log.cc:844] Serialized 78 byte log entry
0709 13:14:00.671117 (+645899us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:00.671175 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.671272 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.283646 (+612374us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:01.376564 (+ 92918us) raft_consensus.cc:1685] finished
0709 13:14:01.376587 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.405829 (+ 29242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63477,"raft.queue_time_us":55884,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8852,"spinlock_wait_cycles":12900992,"thread_start_us":577861,"threads_started":1,"wal-append.queue_time_us":670326}
I20260709 13:14:01.466822  4188 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:01.467147   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 61797.6us
I20260709 13:14:01.467419   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.464354  4213 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:01.473985  3843 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:01.477798  4212 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.478168  3843 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.478456  4212 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:01.479245  3717 consensus_meta.cc:287] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.476s	user 0.019s	sys 0.020s
I20260709 13:14:01.480530  4212 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.480710  4205 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.481426   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.481220  4205 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:01.481799  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50087.1us
I20260709 13:14:01.482060   928 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:01.482744  3955 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:01.483505   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.484210   927 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
W20260709 13:14:01.469069  4285 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:14:01.487149   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:01.469458  4094 consensus_meta.cc:287] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.235s	user 0.014s	sys 0.003s
I20260709 13:14:01.480530  3843 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:01.487823  4127 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.938s	user 0.014s	sys 0.009s
I20260709 13:14:01.466854  4083 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.467509  4250 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.471738  4284 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:01.499475   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.505998   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.507828   927 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:01.467422  4188 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.512739   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:01.513479  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 323236us
I20260709 13:14:01.520818   996 raft_consensus.cc:1217] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[6.3-8.4]   Dedup: 8.4->[]
I20260709 13:14:01.522984   996 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [6.3-8.4]
I20260709 13:14:01.526085   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.532753  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.536893   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.545504   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:01.545396   927 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:01.548946   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.543787  4011 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.550211  4011 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.546068   928 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:01.550742   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2036) took 1611 ms. Trace:
I20260709 13:14:01.553768   850 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:01.557778  4250 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:01.558409  4250 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.558830  4250 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:01.557693  4201 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:01.553314   846 rpcz_store.cc:276] 0709 13:13:59.939256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.939798 (+   542us) service_pool.cc:224] Handling call
0709 13:13:59.943585 (+  3787us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:13:59.947330 (+  3745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.032412 (+1085082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.550463 (+518051us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001cc5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:01.550686 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2418372864}
I20260709 13:14:01.554745   896 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:01.554105   923 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
W20260709 13:14:01.564520   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 77669us
I20260709 13:14:01.568743   996 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:01.569474   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.570426   851 raft_consensus.cc:2410] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:01.562090  4213 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.571435   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.572189  4213 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.572713   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:01.573357   851 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.573741  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:01.574410  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:01.579910   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.581666   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.567816  4057 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.588223   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:01.589601  4013 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.005s
I20260709 13:14:01.591104  4188 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.591379  4021 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.596156   997 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:01.594135   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.566148  4253 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:01.598788   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44110.1us
I20260709 13:14:01.602738   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.607719   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.608443   853 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:01.611912   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 105251us
I20260709 13:14:01.602556  3987 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.556614  4056 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.615258  3987 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.615845   847 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:14:01.618115   847 pending_rounds.cc:85] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:14:01.591051  4212 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.580318  4201 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.619657   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:01.621346  1600 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:01.621577  1002 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:01.553947  4284 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.615280   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:01.617452  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:01.570590  4264 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.624470   999 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:01.625257   997 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:01.618546  3717 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:01.609022  4137 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:01.618311   847 pending_rounds.cc:107] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:14:01.627511   846 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:01.632336   928 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:14:01.631120   847 raft_consensus.cc:2889] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.634431  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:01.639336  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:01.639837  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1764) took 1439 ms. Trace:
I20260709 13:14:01.640242  1000 rpcz_store.cc:276] 0709 13:14:00.200817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.204231 (+  3414us) service_pool.cc:224] Handling call
0709 13:14:00.204581 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.408706 (+1204125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.636317 (+227611us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540034f3d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:01.639765 (+  3448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766647680}
I20260709 13:14:01.640910  4267 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:01.641352  4267 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:01.641805  4267 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:01.643859  4142 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.644469  3803 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:01.645169  3803 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.646214  4142 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.646855  4011 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.647276  3803 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.647944  3717 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:01.634610  4137 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.649294  1598 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:01.649000  3717 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.605330  4013 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.627207  4246 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:01.660995   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:01.616019  4021 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.665021   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:01.667426  1002 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:01.668180   846 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:01.669467  4292 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:14:01.618393  4253 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:01.671558  4253 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.673246  4013 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.673677  4142 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:01.674175  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:01.674786   999 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:01.675236  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.675391   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.676059  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.678354  4201 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.638360  4212 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.680240  4057 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.681548  4212 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.681833   999 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:01.683032  4056 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:01.686085  4294 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:14:01.688522  4171 consensus_meta.cc:287] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.335s	user 0.020s	sys 0.000s
I20260709 13:14:01.688993  4171 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:01.690275  4137 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:01.691177  4171 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:01.691764  1601 raft_consensus.cc:2368] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:14:01.675997   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.663925   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:01.785332  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.732630   997 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:01.785004   999 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:14:02.036800   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.840231  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:01.989019   851 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:02.077404   853 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:01.784721   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.077641   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:02.074832  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.080386   851 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.070894  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:01.868232  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:02.085455  1592 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:02.085393  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.086061  4057 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:02.065662  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.086552  4057 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.086640  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:01.732928  4056 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.087021  4057 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.088171  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.088960  1593 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:02.090931   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1742) took 1313 ms. Trace:
I20260709 13:14:02.091329   847 rpcz_store.cc:276] 0709 13:14:00.777258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.777398 (+   140us) service_pool.cc:224] Handling call
0709 13:14:00.777780 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.711258 (+933478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.090651 (+379393us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54004092d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:02.090858 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907291392}
W20260709 13:14:02.092288   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 460027us
W20260709 13:14:02.092968   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 458647us
I20260709 13:14:02.094275   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:02.096728  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.097404   995 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:02.096306  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:02.084832  1000 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:01.721278  4137 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.380s
I20260709 13:14:01.723503  4260 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.882449  4013 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.102474   846 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:01.694406  4205 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.101372  4201 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:02.107684  4201 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.108140  4260 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.109915  4201 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.110302  4260 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.110093  4057 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.111066   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:02.111425   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.719741  4045 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.083462  4264 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.118122  1598 raft_consensus.cc:2393] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 4: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:02.044813  4021 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.114140   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:01.881458  4274 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:01.787986  4258 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.111720   847 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:14:01.744225  4284 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:01.881872  4221 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.119856   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:02.078848  4255 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.141920  4057 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.142256  4206 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:02.142767  4206 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.142764  4057 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.149966   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:02.150573   847 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.147704  4206 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:02.114610  4171 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.150588  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.156827   846 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:02.163650  4221 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.164994   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:02.166083  4221 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.139029  4201 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:02.167682   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.167956  4201 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.168635  4045 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.170408  4201 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.170990  4201 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:02.177763   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:02.150275  4274 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.180164  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.174787  1001 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 4.
I20260709 13:14:02.171819  4255 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:02.177641  4246 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:02.171372   848 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 3 to 7, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:02.197772   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:02.131722  4253 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:02.171523  4201 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.199985  4253 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.201375  4201 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.182518   847 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:02.201985  4253 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.202852   847 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:02.171773  4123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51112 (request call id 77) took 4625 ms. Trace:
I20260709 13:14:02.207197   851 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:02.206415  4123 rpcz_store.cc:276] 0709 13:13:57.546470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.546659 (+   189us) service_pool.cc:224] Handling call
0709 13:13:57.921731 (+375072us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b1800044520. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b7a73d 00007f0d02b45f3f 00007f0d02b456b3 00007f0d02b3d9fa 00007f0d07b054c9 00007f0d01081c54 00007f0d01081be2 00007f0d01081b11 00007f0d01081a8c 00007f0d01080d42
0709 13:13:59.474110 (+1552379us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5800840ed0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02a9b0aa 00007f0d02b1d447 00007f0d02b3a70a 00007f0d02b3b76a 00007f0d02b3a91b 00007f0d02b4603c 00007f0d02b456b3 00007f0d02b3d9fa 00007f0d07b054c9 00007f0d01081c54
0709 13:13:59.484247 (+ 10137us) log.cc:844] Serialized 494 byte log entry
0709 13:14:02.156539 (+2672292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4466,"raft.run_cpu_time_us":2151,"raft.run_wall_time_us":93927,"spinlock_wait_cycles":1566537856,"thread_start_us":98545,"threads_started":1,"wal-append.queue_time_us":216011,"wal-append.run_cpu_time_us":1987,"wal-append.run_wall_time_us":1051941}
I20260709 13:14:02.187964   846 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:02.205952  4274 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.228741   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7cf198f3795b43ff951a11167bc21580 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:14:02.233559   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.234289   996 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:02.234725  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.234844   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:02.235229   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.182722  4056 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.240736  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:02.128703  4013 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.249478   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:02.208745  4171 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.258126   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.259212   997 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:02.147024  4258 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.267639   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2054) took 1105 ms. Trace:
I20260709 13:14:02.270691   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:02.268078   855 rpcz_store.cc:276] 0709 13:14:01.161976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.648907 (+486931us) service_pool.cc:224] Handling call
0709 13:14:01.649183 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.649378 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:01.816399 (+167021us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.267283 (+450884us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.267339 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.267381 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.267445 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.267560 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":509391,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":558410,"threads_started":2}
I20260709 13:14:02.253561  4023 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.260795  4123 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:02.266055  4213 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.693s	user 0.003s	sys 0.008s
I20260709 13:14:02.337487   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:02.280747  4123 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.284129  4056 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:02.310745  4267 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.000s
W20260709 13:14:02.344549   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 89869.6us
I20260709 13:14:02.339728  4023 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.309767  4246 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.343731  4123 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.341418  4045 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.354809  4253 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.351918  4267 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.366041  3987 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:02.416705   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 133173us
I20260709 13:14:02.414748   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2053) took 1253 ms. Trace:
I20260709 13:14:02.381291  4011 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.422053   852 rpcz_store.cc:276] 0709 13:14:01.161650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.646778 (+485128us) service_pool.cc:224] Handling call
0709 13:14:01.647041 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.647311 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:02.066577 (+419266us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.414357 (+347780us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.414429 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:02.414477 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.414541 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.414684 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":489370,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2095,"thread_start_us":712630,"threads_started":2}
I20260709 13:14:02.378419  4246 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:02.435690   821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.610s (SENT) [suppressed 29 similar messages]
I20260709 13:14:02.428887  4213 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.439278  4250 consensus_meta.cc:287] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.878s	user 0.010s	sys 0.004s
W20260709 13:14:02.455826   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 67887.1us
I20260709 13:14:02.456919   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:02.458110   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 786332us
I20260709 13:14:02.438931  4171 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:02.453780   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 92949.9us
W20260709 13:14:02.458681  4305 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:14:02.433548  4045 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:02.459221  4303 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 13:14:02.507193   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 403765us
I20260709 13:14:02.451803  4286 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.509761   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.509579  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.514806   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:02.464745  4142 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:02.510903   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.534036   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:02.535060  4260 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:02.535854  4205 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:02.536744  4205 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:02.537070  4205 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:02.468286   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2056) took 1271 ms. Trace:
I20260709 13:14:02.538417  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.538520  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.541361  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.561112   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.454825  4250 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.540376  4260 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:02.462092  4213 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.537911   854 rpcz_store.cc:276] 0709 13:14:01.197284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.657081 (+459797us) service_pool.cc:224] Handling call
0709 13:14:01.670629 (+ 13548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.670918 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:02.036788 (+365870us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.467729 (+430941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.467827 (+    98us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:02.467904 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.468010 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.468196 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":492840,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":211,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":33634,"thread_start_us":712447,"threads_started":2}
I20260709 13:14:02.570329   923 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
W20260709 13:14:02.533197  4257 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:02.572791  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:02.580045  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.573711   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:02.582418   928 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.583932  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.532554  4013 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.570618   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 430) took 1019 ms. Trace:
I20260709 13:14:02.592430   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.593070   996 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:02.596175   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 433) took 1037 ms. Trace:
I20260709 13:14:02.600872  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.585278  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:02.601647  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:02.601694  1593 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.578622   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2060) took 1162 ms. Trace:
I20260709 13:14:02.597919   847 rpcz_store.cc:276] 0709 13:14:01.558438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.226487 (+668049us) service_pool.cc:224] Handling call
0709 13:14:02.236706 (+ 10219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.236943 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:02.279141 (+ 42198us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.595026 (+315885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.595128 (+   102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:02.595234 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.595369 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.596095 (+   726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26093,"prepare.queue_time_us":30957,"prepare.run_cpu_time_us":7984,"prepare.run_wall_time_us":44260,"thread_start_us":232896,"threads_started":1}
I20260709 13:14:02.601047  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:02.606318  1004 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.606566  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:02.607810  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:02.608381  1597 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.611756   929 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:02.616339  4057 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.616883  4057 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.618120  1593 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:02.619199  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:02.620632  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:02.621330  1003 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.632941  1597 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:02.635447   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1779) took 1368 ms. Trace:
I20260709 13:14:02.641932   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:02.642565   924 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.642308   849 rpcz_store.cc:276] 0709 13:14:01.266977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.689462 (+422485us) service_pool.cc:224] Handling call
0709 13:14:01.689780 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.690166 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.690190 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.690673 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.096783 (+406110us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:02.506651 (+409868us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.506894 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.506989 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.635227 (+128238us) raft_consensus.cc:1685] finished
0709 13:14:02.635245 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.635387 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41849,"prepare.queue_time_us":67627,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":18578,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":8837,"thread_start_us":795653,"threads_started":2,"wal-append.queue_time_us":588839}
I20260709 13:14:02.636065   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.644629   925 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
W20260709 13:14:02.645586  4304 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
W20260709 13:14:02.646574   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 144718us
I20260709 13:14:02.647773   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:02.649363  4260 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.652042   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:02.647473  1003 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:02.662358  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:02.537927  4205 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.663552  4258 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.640570  4246 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.543903  4286 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.687186   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:02.690846   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 413061us
W20260709 13:14:02.687407   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 251685us
W20260709 13:14:02.698024  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 121590us
I20260709 13:14:02.604111   848 rpcz_store.cc:276] 0709 13:14:01.416202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.129565 (+713363us) service_pool.cc:224] Handling call
0709 13:14:02.146166 (+ 16601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.154553 (+  8387us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:02.208226 (+ 53673us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.578257 (+370031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.578318 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:02.578364 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.578430 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.578552 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61749,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8266,"thread_start_us":369504,"threads_started":2,"wal-append.queue_time_us":239737}
I20260709 13:14:02.698782  1598 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:02.679105   924 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:02.713191   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.677537  4142 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:02.715574  4306 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:14:02.766132   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:02.787343   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.590915   846 rpcz_store.cc:276] 0709 13:14:01.553209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.210090 (+656881us) service_pool.cc:224] Handling call
0709 13:14:02.229100 (+ 19010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.241232 (+ 12132us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:02.241538 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.570252 (+328714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.570309 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:02.570353 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.570422 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.570558 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29419,"prepare.queue_time_us":21777,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":54001,"thread_start_us":290040,"threads_started":1}
I20260709 13:14:02.702967  4258 raft_consensus.cc:2764] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:14:02.686929  3717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.015s	sys 0.006s
I20260709 13:14:02.765017  1004 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:02.793140   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.793803   923 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
W20260709 13:14:02.710362  4310 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:14:02.718936  4246 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.717376  4255 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:02.808136  4255 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:02.809201  4255 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:02.810238  4255 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.725517  4057 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:02.703591  4267 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.717056  4123 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.813349  1001 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:02.697753  4307 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:02.714522  4286 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.726539  4023 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.815726  4309 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:14:02.800390   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:02.805634  4201 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.817001  4309 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:02.817369  4309 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:02.817480  4201 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:02.817425   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:02.818140  4201 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:02.818692  4201 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:02.818351  4309 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:02.818976   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:02.829747  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.807565  4319 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.805892  4315 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:02.839457  4315 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:02.830670  1597 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:02.792943  3717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.016s	sys 0.007s
I20260709 13:14:02.840086  4315 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:02.844205  1597 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:14:02.845162  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:02.853456  1593 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:14:02.863907   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 108842us
I20260709 13:14:02.847508  4201 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.013692   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.861924   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:02.790472  4142 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:02.869490  4206 consensus_meta.cc:287] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.003s
I20260709 13:14:02.929397   894 leader_election.cc:400] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:14:03.002583  4213 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:02.869848  4189 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.856051  4322 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:02.942826   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:02.855715  3803 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:14:02.953402   928 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:02.856843  4319 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.003142   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:02.856257  4315 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:03.013118   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 308502us
I20260709 13:14:02.877864  4045 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.017825   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:03.024880   925 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:14:03.017791  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 259312us
I20260709 13:14:03.018591   894 leader_election.cc:403] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:14:02.853382  4309 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:03.019328   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 157482us
W20260709 13:14:03.019979   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 201047us
I20260709 13:14:03.029388   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:03.036983  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 264616us
W20260709 13:14:03.041997   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 186854us
I20260709 13:14:03.045318  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:03.048018  4317 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:03.048578  4317 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.048929  4317 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:03.049558  4329 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:14:03.050568  4171 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:03.051066  4171 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:03.051843   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 109123us
I20260709 13:14:03.052121  4171 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.053121   821 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.023516  4322 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:03.054304  4322 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.039738   969 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:03.024369  4213 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:03.053850  4171 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:03.057794   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:03.057878   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.061364   924 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:03.061647   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.063563   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.021744  4206 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.111145   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 463536us
I20260709 13:14:03.118459  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:03.120038  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:03.120803  1597 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:03.123379  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57712.5us
I20260709 13:14:03.127478  1600 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:03.059391  4011 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.060570  4213 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:03.068068  4056 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:03.032028  4189 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.067008  4329 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:03.142256  4250 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.144912  4052 consensus_queue.cc:1059] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:03.147365  4206 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.065958  4326 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:03.152040  3803 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.152871  4329 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:03.136142  4213 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:03.156984  4255 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.149415  4315 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:03.137501  4056 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.148094  4011 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.195065   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.170969  4307 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:03.172156  4326 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:03.194154  4056 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:03.194705  4201 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:03.218490   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 165425us
I20260709 13:14:03.172839  3803 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.222455  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 161432us
I20260709 13:14:03.192533  4255 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.226188  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:03.213284   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 183938us
I20260709 13:14:03.216424  4045 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:03.236179   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.232344   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 192679us
I20260709 13:14:03.234158   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.241715  4011 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.242194  4255 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:03.217989  4337 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:03.245142  1564 leader_election.cc:400] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
I20260709 13:14:03.245534  1564 leader_election.cc:403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:03.244925  4013 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:03.248283  4013 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.248638  4013 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:03.247694  4123 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.264072  4056 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.264979  4307 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.266153  4189 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.232936  4142 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.266942  4056 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.267908  3803 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.263885  4326 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:03.269910   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 99112.5us
I20260709 13:14:03.270066   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:03.230715  4201 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:03.264585  4258 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:14:03.275837   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.282478  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.279739   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:03.284219  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 159889us
I20260709 13:14:03.296599  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.244067  4331 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:03.300908  4319 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.301115   855 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:03.246546  4255 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.303514   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.303715   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.280618  4011 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.281044  4258 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:03.291224  4189 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:03.290369   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 56348.8us
I20260709 13:14:03.280903  4142 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:03.375145  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 72332.7us
I20260709 13:14:03.378342   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1781) took 2110 ms. Trace:
I20260709 13:14:03.277467  4274 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
I20260709 13:14:03.379258  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.280772  3803 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:03.369941  4354 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:14:03.310568  4327 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.243s	user 0.001s	sys 0.001s
I20260709 13:14:03.388849  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:03.395000  1601 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:03.378955   850 rpcz_store.cc:276] 0709 13:14:01.267507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.697727 (+430220us) service_pool.cc:224] Handling call
0709 13:14:01.756527 (+ 58800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.870406 (+113879us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:02.579032 (+708626us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.168884 (+589852us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.238985 (+ 70101us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.252580 (+ 13595us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b1800067a80. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:03.371660 (+119080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:03.371763 (+   103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:03.371824 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.372100 (+   276us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.378276 (+  6176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23740,"prepare.queue_time_us":210368,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":119,"raft.queue_time_us":253950,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":64905,"spinlock_wait_cycles":4838656,"thread_start_us":1143517,"threads_started":4,"wal-append.queue_time_us":635359}
W20260709 13:14:03.343122  4328 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:14:03.391747   924 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:03.387888   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.343672  4253 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:03.417757   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.418507   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:03.419080   928 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:03.309902  4351 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.421190   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:03.277040  4329 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.379782  4331 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:03.310945  4335 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:03.424809   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:03.397482  4246 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.006s
I20260709 13:14:03.384205  4201 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:03.307963  4255 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.448951   928 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:03.451903   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 150158us
I20260709 13:14:03.426463  4341 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:03.442179  4335 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.458338   924 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:03.377853  4011 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:03.463294   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 178135us
I20260709 13:14:03.426100  4253 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:03.397462  4189 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:03.309381  4213 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.443727   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.401458  3803 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.470790  4344 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:03.426697  4332 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:03.441141  4343 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:03.477363  4341 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.450068  4326 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:03.456308  4246 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:03.339838  4337 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:14:03.452277  4319 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:03.471238  4331 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.505033  4253 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:03.384508  4142 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.522414   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 78736.9us
W20260709 13:14:03.523705   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 244101us
I20260709 13:14:03.516659  4352 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.528644   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.520092  4332 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.540807   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:03.524806   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.456007  4351 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.542186   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.520323  4344 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:03.505815  4335 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:03.465253  4201 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.536074  4330 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:14:03.527546  4341 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:03.552022  4363 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:03.554939  4366 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:03.555451  4356 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:03.535532  4319 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.533897  4353 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:03.544137  4343 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:03.499084  4258 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.560089   969 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:03.546465  4332 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:03.588840  4340 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.353s	user 0.005s	sys 0.000s
I20260709 13:14:03.557294  4315 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:03.580809  4320 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:03.630342   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 404) took 1396 ms. Trace:
I20260709 13:14:03.549089  4352 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.580233  4342 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:03.584386   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67505.6us
I20260709 13:14:03.551295  4344 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:03.633689   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 220147us
I20260709 13:14:03.565716  4319 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:03.630769   998 rpcz_store.cc:276] 0709 13:14:02.233808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.235516 (+  1708us) service_pool.cc:224] Handling call
0709 13:14:02.261474 (+ 25958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.593592 (+332118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.593627 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.606327 (+ 12700us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.017335 (+411008us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:03.224854 (+207519us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.235708 (+ 10854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.237321 (+  1613us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.630130 (+392809us) raft_consensus.cc:1685] finished
0709 13:14:03.630150 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.630281 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10531,"prepare.queue_time_us":544678,"raft.queue_time_us":81215,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":8878,"thread_start_us":811024,"threads_started":3,"wal-append.queue_time_us":529504}
W20260709 13:14:03.636261   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 330981us
W20260709 13:14:03.633878   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 91935.7us
I20260709 13:14:03.642586   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.543195  4326 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.654425  4359 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:03.654858  4359 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.655194  4359 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:03.655231  4353 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:03.656155  4353 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:03.656483  4353 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:03.657267  4353 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.658418  4353 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:03.649554  4320 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:03.660127  4255 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.661123  4320 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.663829  4356 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:03.664268  4356 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:03.664845  4201 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.666684  4142 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.667268  4142 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.667797  4142 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:03.674888  4357 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:14:03.676630  4358 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:14:03.678033  4343 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.665091  4356 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.682277   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 157570us
I20260709 13:14:03.652768   818 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.635455  4349 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:03.635217  4342 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:03.691936   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93571.8us
W20260709 13:14:03.660123  4360 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:03.669932  4142 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.651947  4337 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:03.692159  4317 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.000s
I20260709 13:14:03.627365  4189 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.693671  4353 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:03.723163   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75029.6us
W20260709 13:14:03.710899  4322 consensus_meta.cc:287] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.001s
I20260709 13:14:03.695539  4342 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:03.726784   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.726027  4370 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:03.748986  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.725762  4356 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:03.703387  4349 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.769640  1601 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:03.779006   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:03.738391  4353 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:03.739676  4213 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.719628  4317 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.760784  4201 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:03.740067  4322 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.789563  4274 raft_consensus.cc:2764] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:03.789703  4363 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.789688  4315 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:03.792268  4371 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 13:14:03.824496  4365 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:14:03.826431  4317 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.795392  4320 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:03.818267  4363 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.827893  4320 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.830021  4320 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.830595  4320 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:03.830147  4363 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.752007  4370 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.824496  4353 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.833860  4370 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:03.835906   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83243.5us
I20260709 13:14:03.836948   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.838421   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.794708  4366 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:03.840128  4326 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.840332  4366 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.841185  4349 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.845906  4344 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:03.847769  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46536us
I20260709 13:14:03.849658  4352 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:03.853315  4013 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.001s
W20260709 13:14:03.853919  4171 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.000s
I20260709 13:14:03.853881  4013 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.857882  4322 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.847899  4351 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.830964  4320 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:03.838887  4253 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.866779   924 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:03.866077  4056 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:03.820294  4359 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:03.884105  4309 consensus_meta.cc:287] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.003s
I20260709 13:14:03.884899  4359 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:03.886212   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:03.886638  4253 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:03.887035   927 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:03.887557  4342 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:03.888288  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:03.870581  4315 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:03.804776  4213 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:03.871436  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 240833us
I20260709 13:14:03.810612  4011 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:03.910845   895 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.911725   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.911931   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.912459   998 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:03.909022  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.937069   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.938016   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:03.938341   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.938908   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.939175   846 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:03.939699   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.940264   854 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:03.938031   855 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:14:03.941872  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 93190.3us
I20260709 13:14:03.870759  4329 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:03.944921  1001 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:03.945973   848 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:03.946648   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.947633   849 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 7: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:03.948266   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.943197  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.950531  1002 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:14:03.949033   999 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:03.952721  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:03.944900  4212 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:03.960728  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:03.957008  4315 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:03.964092   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.965718   997 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:03.964025  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.966871   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:03.977202   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.061004   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:03.966570  1000 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
W20260709 13:14:03.964479   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 219071us
I20260709 13:14:04.226979   855 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:04.182240  4013 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.228366   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:04.229157  4309 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:04.231472   821 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:04.102263  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 135935us
I20260709 13:14:03.912456  4213 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.239123   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.238874   850 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:04.158440   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.241528   995 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:03.959738  4286 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:03.963984  4343 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:04.167225   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 256460us
I20260709 13:14:03.963972  4336 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:03.916587  4332 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:04.178411  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.072470  4342 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.249336  4213 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.251123  4332 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.179688   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.252066   849 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:04.065912  4253 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:04.252921  4253 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.254951  4253 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:04.186831   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:04.193274  4319 consensus_meta.cc:287] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.004s
I20260709 13:14:04.256224   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.256376  4319 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.258512  4319 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:04.259181  4309 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:04.260102  4309 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:04.260660  4309 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:04.260718  4246 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:04.261089  4309 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:04.262199  4309 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.264492   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.264768  4344 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.265120   996 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.265301  4309 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:04.204444  4376 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 13:14:04.270866   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:04.271754  4253 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:04.273149  4253 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:04.198549  4335 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.000s
W20260709 13:14:04.212321  4384 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:14:03.954207  4320 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.276993  4335 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.280725   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:04.233568  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.246351  4343 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.263562  1003 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:04.264143   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.283352   997 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:04.199224   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.284916   848 raft_consensus.cc:2410] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:04.287149  4013 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:04.291311   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.292078   998 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.294660 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:04.300881  4320 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:04.301123   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:04.302038  4332 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:04.302557  4332 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:04.303376  4332 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.309389  4205 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:04.309870   749 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 119) took 13625 ms (client timeout 15000 ms). Trace:
I20260709 13:14:04.312657  4329 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:04.313651   995 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:04.314687  4013 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:04.316087  4343 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:04.319525   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48730.1us
W20260709 13:14:04.310384   749 rpcz_store.cc:269] 0709 13:13:50.684915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.685297 (+   382us) service_pool.cc:224] Handling call
0709 13:14:04.309667 (+13624370us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:00.407695 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:00.460262 (+ 52567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:00.460304 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:14:00.460327 (+    23us) tablet.cc:662] Decoding operations
0709 13:14:00.461383 (+  1056us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:00.461443 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:00.461472 (+    29us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:14:00.668224 (+206752us) tablet.cc:701] Row locks acquired
0709 13:14:00.668249 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:14:00.668441 (+   192us) write_op.cc:270] Start()
0709 13:14:00.668598 (+   157us) write_op.cc:276] Timestamp: P: 1783602840668386 usec, L: 0
0709 13:14:00.668622 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:14:00.669317 (+   695us) log.cc:844] Serialized 3636 byte log entry
0709 13:14:01.282257 (+612940us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:01.635600 (+353343us) op_driver.cc:464] REPLICATION: finished
0709 13:14:02.276697 (+641097us) write_op.cc:301] APPLY: starting
0709 13:14:02.335613 (+ 58916us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:02.351155 (+ 15542us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:02.351197 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:02.457310 (+106113us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:02.459675 (+  2365us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:14:02.463596 (+  3921us) write_op.cc:312] APPLY: finished
0709 13:14:02.619939 (+156343us) log.cc:844] Serialized 71 byte log entry
0709 13:14:02.970858 (+350919us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:03.021393 (+ 50535us) write_op.cc:454] Released schema lock
0709 13:14:03.031628 (+ 10235us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:03.132627 (+100999us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:03.134749 (+  2122us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":468660,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":579558,"apply.run_cpu_time_us":23749,"apply.run_wall_time_us":799027,"num_ops":9,"prepare.queue_time_us":255424,"prepare.run_cpu_time_us":24931,"prepare.run_wall_time_us":411469,"raft.queue_time_us":164171,"raft.run_cpu_time_us":17026,"raft.run_wall_time_us":447602,"replication_time_us":947759,"spinlock_wait_cycles":2861824,"thread_start_us":902998,"threads_started":5,"wal-append.queue_time_us":598365,"wal-append.run_cpu_time_us":29342,"wal-append.run_wall_time_us":1699981}]]}
I20260709 13:14:04.279333  4335 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:04.332528  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 99065.1us
I20260709 13:14:04.323045   850 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
W20260709 13:14:04.315186  4356 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.000s
W20260709 13:14:04.208364  4341 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.006s
I20260709 13:14:04.320779   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:04.314798  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:04.342212   997 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:04.344013  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:04.340076   998 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:04.341514   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:04.345834   924 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:04.338555   996 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:04.350775  1598 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:04.351944   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.357825  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.358336  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.360284   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:04.332634  4343 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.361593   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:04.362170   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 116028us
I20260709 13:14:04.364333   928 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:04.366394   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:04.364300  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.371263  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:04.364985  4344 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:04.384225  4344 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.389102  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:04.390223  1594 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:04.391947   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:04.392625   924 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.395462  4326 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.391040  4253 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:04.396143  4326 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.411487  4356 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.412322  4356 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.413105  3987 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.413504  3987 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.413836  3987 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.414537  4356 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:04.412583   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46261.1us
I20260709 13:14:04.416229   969 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:04.417105   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:04.417960  4319 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.418347  4319 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:04.418715  4319 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.420976  4343 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.422667   848 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:14:04.423810   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 122783us
I20260709 13:14:04.425312   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.426070  4205 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:04.426477   895 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.427349  4205 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:04.350451  4341 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.429661  4341 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.357365  4013 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.430421  4013 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.430778  4013 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:04.433911   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 72391.1us
I20260709 13:14:04.434674   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:04.435366  4253 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:04.436093  4253 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:04.391466   854 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:04.457511   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:04.387167  4344 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.467064   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:04.423414  4343 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.364703  4352 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:04.371176  4337 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:04.490666  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.491556  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:04.537948   924 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:04.564112   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.565384   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.566236   996 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:04.483946  4205 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:04.570865   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.571473   998 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:04.486744  4331 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:04.490970  4142 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:04.587740   997 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:04.590730   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.594053   997 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:04.592077  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.594640   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.594786  1594 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:04.593680  4205 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.543900  4356 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:04.595583   995 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:04.597791   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.601147   996 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
I20260709 13:14:04.600725  4205 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.628373   968 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:04.537948  4344 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.635221  4349 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:04.636096  4338 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:04.643877  4338 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.646306  4338 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.528012  4368 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
W20260709 13:14:04.642599   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:04.648284   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.609627  3987 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.636317  4352 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.636585  4253 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.598039  4315 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:04.589148  4331 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:04.635342  4343 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:04.635993  4341 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:04.650995   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:04.640372  4045 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:04.656077   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:04.635787  4381 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:04.636823  4344 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.609138  4013 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.601668  4356 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.652350  4386 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:04.649852  4171 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:04.665597  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.651356  4286 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.651552  4371 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:04.658012  4023 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:04.655071  4331 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:04.636014  4349 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.684744  4356 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.682215  4286 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.658646  4253 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.687013  4286 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.660828  4352 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.665632  4344 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.690552   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.665514  4205 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:04.694084   998 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:04.693210  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.669248  4386 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.697841   855 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
W20260709 13:14:04.702096   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 61710us
I20260709 13:14:04.681248  4023 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:04.665398  4381 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.690526  4352 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.673545  3987 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:04.707983  4315 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:04.709000  4315 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:04.709350  4315 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:04.710244  4315 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.711488  3803 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:04.699096  4386 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:04.687556  4326 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.687657  4253 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.672143  4171 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.665561  4341 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.669129  4343 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.713577   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.682711  4371 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.700472  4205 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:04.725394   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:04.726544  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.728154  4142 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:04.728771  4343 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.736581  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.731050  4315 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.734045  4326 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:04.733351  4142 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:04.724246  4319 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.763247  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.763811   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:04.753695   847 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:04.747092  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.765106  4286 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:04.765132   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.765944   851 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:04.768538   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:04.765520  4322 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.771524  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.772503  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.773198  1004 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.775223  4349 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.778402  4319 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:04.779821  4171 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:04.744856  4013 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.796384   851 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:04.801579  1000 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:04.801545  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:04.803807   995 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:04.803879  4341 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.807978   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.808398  4013 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:04.808890   998 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:04.809832  4056 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.810603  4056 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.811704  4326 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:04.812922  3926 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:04.770865  4329 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:14:04.813304  3926 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:04.814085  4258 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.814287  3926 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.815207   996 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:04.799317  4322 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.809911  1001 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:14:04.819860   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:04.824441   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:04.825084   853 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.827723  4246 raft_consensus.cc:2764] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:14:04.835112  4392 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:04.834949  4348 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:04.836898  4322 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.838840  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:04.839375  4391 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:14:04.839790  4286 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:04.841086  4056 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.839177  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:04.842480  4381 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:04.796844  4343 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.843138  4348 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:04.843480  4343 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.843961  4348 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:04.845278  4348 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.845923  4343 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.833079  4371 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:04.830883  4205 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.855438  1004 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:04.862331  3926 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.773329  4274 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
W20260709 13:14:04.868940  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44939.9us
I20260709 13:14:04.879299  1567 leader_election.cc:400] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
W20260709 13:14:04.946164   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.879179  4255 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.996850  4255 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:04.879470  4346 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:14:04.878389  4344 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.742004  4023 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:04.882418  4013 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:14:05.009840  4344 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:05.010485   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 180534us
I20260709 13:14:04.976639   853 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:05.010927   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.880925  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:04.982546  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:04.865720  4246 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:14:04.985705  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.990319  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.013056  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.993476   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:04.994810   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.995438  4356 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:05.014894   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:04.996196   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:04.969797  4337 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:04.999297  4255 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:05.000916   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:05.001344   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 177156us
I20260709 13:14:05.001464   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:05.002471  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:05.007697   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.011937   997 raft_consensus.cc:2368] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:14:05.013124  1003 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:04.994956  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:05.016440  1000 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:05.028203   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:05.025588   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:05.039287   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.040032   854 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.040095  1001 raft_consensus.cc:2368] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:05.043107  4319 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:05.043999  4255 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:14:05.044337  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 163469us
I20260709 13:14:05.044878  4364 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:05.045158  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:05.045563  4364 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:05.046361  1567 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:05.044418  4255 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.047942  4336 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:05.048396  4336 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.048846  4336 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.048463  4331 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.048692  4255 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.050130  4255 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.042678   751 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:05.054340   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.054720  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:05.055059   850 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:05.056803   846 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:05.057472  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.056725   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.058861   855 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.060796   850 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:05.062310   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.063460   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.064775   850 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:05.065083  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 186169us
I20260709 13:14:05.066332  1003 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:14:05.063349   852 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:05.031986  4045 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:05.069497   853 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:05.034495  4344 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:05.074558   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.077837   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.046792  4364 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.052707  3803 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:05.039754  4356 consensus_queue.cc:1243] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:14:05.079916   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54434.8us
I20260709 13:14:05.088454   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:05.079411   924 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.084712  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:05.087198   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.086442   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.098480   924 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:05.101954  4326 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:05.106165   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:05.111521  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.109241  4329 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:05.070132  4386 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.121032   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.122319   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.122656  4386 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:05.123425   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.124022   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.124457  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:05.132480  1003 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:05.083895   751 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 5 to 6, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:05.110292  4142 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.080698  4189 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:05.143352   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.144315   850 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:05.085855  4394 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:05.146154   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.145035   854 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:05.158270   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:05.158442  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.093698  4123 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:05.157038  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.160020  1001 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:05.146698   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.161039   850 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.149381   855 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:05.084666  4045 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:05.169605   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.170354   851 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:05.078575  4359 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:14:05.167313   854 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:05.164491   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.168745  4189 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:05.124435  4356 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.173346   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.174814   855 raft_consensus.cc:2410] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:14:05.134186  4329 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:05.178299   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 96318.4us
I20260709 13:14:05.188046   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.179561  1003 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:05.180485   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:05.180853   846 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:05.190034   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.190423   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.186734   751 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 5, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:05.191409   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.192437   925 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:05.186679   850 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:05.187498  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.193482  1001 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:05.188627   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.195451  4359 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:05.195957  4359 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.196385  4359 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.204018  4346 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.204617  4346 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.209583   854 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.175227  4123 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.208145   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.217077   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.206496   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.178895  4284 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.229000  1001 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:05.229434   751 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 3 to 5, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:05.176277  3926 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:05.172791  4246 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:14:05.179216  4394 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:05.263787   929 raft_consensus.cc:2393] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:05.249950   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.262727   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.271384  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.271960   851 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:05.262313  4375 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:05.207335  4393 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:05.283751  1004 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:05.284229  1004 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.258423   854 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:05.268844   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.286023   921 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 5: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:05.286806   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.280886   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:05.186466  4356 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.295466   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:05.296130   751 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:05.311358  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.289358  3926 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:14:05.313160   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.313925   929 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.321717  4023 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.325906  4307 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:05.326377  4307 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:05.328040   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.328696   921 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:05.330555  4399 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:14:05.332441  4405 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:14:05.339998  4331 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:05.327401  4307 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.309162  1004 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.348506  4142 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:05.353060  4336 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.343015   751 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 5 to 6, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:05.354918  1004 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:05.357467  4123 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.259528  4403 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:14:05.368968  4329 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.306438  4393 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.292301  3803 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:14:05.386586  4366 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:05.387135  4366 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.388108  4366 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.389225  4366 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:05.396235  4346 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:05.397703  4407 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:05.400591   927 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:05.400591   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.401809   922 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:05.401997   926 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:05.402412   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.401901   930 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:05.402957   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.403951   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.404254   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.404557   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.369501   751 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 4 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:05.413012   921 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:05.413928   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.418260   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.403019   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.418182   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.281208  3987 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:05.428910   751 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 1 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:05.422406  1004 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:05.404595   928 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:05.418968   922 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
W20260709 13:14:05.431488   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 229109us
I20260709 13:14:05.420109   930 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:05.403036   927 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:05.472674  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:05.473373  1597 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:05.265792  4284 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.488190  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.488824   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:05.309680  4189 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.489116  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.489384  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.489809  1001 raft_consensus.cc:2368] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:05.488866  4284 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:05.289095  4045 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:05.491464  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 74543.3us
I20260709 13:14:05.494685  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.494813  1597 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:05.486258   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 8, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 8 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:05.471843  4258 consensus_meta.cc:287] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.002s
I20260709 13:14:05.512061   821 leader_election.cc:400] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:14:05.532289   821 leader_election.cc:403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:05.527530   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.454159  3987 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:05.472620  4171 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:14:05.529783   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.549151  4393 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:05.559719   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.579440   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.580788   921 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:05.582350   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.634136   928 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:05.602049  1601 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:05.583273   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:05.533619   927 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:14:05.483624  3803 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:05.660799   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.662799   921 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:05.619712  3987 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:05.334657  4394 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.618680  4336 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:05.646384  4367 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:05.685801   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 392702us
I20260709 13:14:05.618530  4406 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:05.648283  3803 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.695400   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:05.709823   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:05.697810   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.711735  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.707849   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.673558  4375 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.726020   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.726373   968 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:05.747550   925 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260709 13:14:05.747978   925 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:05.755478   924 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:05.765636   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.711220  4142 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
W20260709 13:14:05.784158   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 289088us
W20260709 13:14:05.768613  4315 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.025s	user 0.011s	sys 0.000s
I20260709 13:14:05.782668   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.789024   925 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:14:05.722494  4331 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:14:05.786652   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.715777  4367 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.788724   921 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:05.726890  4406 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:05.789899  4359 consensus_meta.cc:287] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.003s
I20260709 13:14:05.827390  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.829099   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.799401   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:05.833693  1004 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:05.835331  1002 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:05.836067  1003 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:05.791718  4386 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:05.798612  4255 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.748s	user 0.014s	sys 0.000s
I20260709 13:14:05.840477   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:05.840160   997 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:05.852487  4420 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:14:05.880179   929 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:14:05.880760   929 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:05.906599  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.907559  1003 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:05.856383  4023 consensus_meta.cc:287] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.003s
W20260709 13:14:05.909693   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 326504us
I20260709 13:14:05.791802   854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:05.913128   854 raft_consensus.cc:606] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:05.913446   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 187190us
I20260709 13:14:05.910022   925 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:05.914379   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.914606   854 raft_consensus.cc:614] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:05.914529   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:05.896500  4413 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:05.918095   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:05.920562   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.896693  4403 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.787071  3803 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.921630   921 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
W20260709 13:14:05.916049  4396 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:14:05.911296  4423 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:05.834074  4359 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.909266  4284 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:05.949468  4284 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:05.950649  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:05.952001  4424 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9.
I20260709 13:14:05.924178  4403 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:05.952327   929 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:05.952723  4424 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:05.953342  4424 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:05.953512   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.953737  4424 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.954144  4424 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:05.954314   929 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:05.954882   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.955155   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.956281   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.973345  1004 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:05.975117  1004 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:05.897923  4045 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:14:05.921587  3926 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:05.845808  4375 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:05.940800  4417 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:14:05.940953  4419 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:14:05.941908  4359 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:05.942143  4418 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:14:05.942693  4395 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:14:05.923113  4413 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.948658  4406 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:05.950456  4284 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:05.929347   997 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:05.960054   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:05.896008  4332 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:14:05.970362   854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
W20260709 13:14:05.958019  1601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 372) took 1615 ms (client timeout 2126 ms). Trace:
I20260709 13:14:05.959192   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.012624   854 raft_consensus.cc:606] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:05.953364   930 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:14:05.983569  1597 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:05.985142   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:05.988152   926 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.915709  4402 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:14:06.001225  4205 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:05.972615   924 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
W20260709 13:14:05.918705   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 192608us
I20260709 13:14:06.087029   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 463) took 1248 ms. Trace:
I20260709 13:14:06.142282  4329 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:06.086030   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.099153  4332 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:06.154289  4332 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:06.143491   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.144695  4205 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:05.985870  3926 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:06.141692   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 352501us
I20260709 13:14:06.145244   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.082566  4428 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.145963  4386 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:06.012575  1601 rpcz_store.cc:269] 0709 13:14:04.343003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.357509 (+ 14506us) service_pool.cc:224] Handling call
0709 13:14:05.957830 (+1600321us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400700154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:05.957933 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859424512}
I20260709 13:14:06.121189   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.180647   922 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:06.130405   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.142316   847 rpcz_store.cc:276] 0709 13:14:04.838381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.841649 (+  3268us) service_pool.cc:224] Handling call
0709 13:14:04.842079 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.854960 (+ 12881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.855007 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:04.865640 (+ 10633us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.083512 (+217872us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:05.083556 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.084388 (+   832us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.086808 (+1002420us) raft_consensus.cc:1685] finished
0709 13:14:06.086828 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.086968 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14799,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":2389,"thread_start_us":98936,"threads_started":1,"wal-append.queue_time_us":583959}
I20260709 13:14:06.148510   925 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:06.149590  4315 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:05.972393  4423 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:06.005031  4336 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:06.012905  4413 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.152349  4359 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.018879  4023 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
W20260709 13:14:06.153103   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 235114us
I20260709 13:14:06.143493  4393 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:06.040030  4045 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.130s
I20260709 13:14:06.013945   854 raft_consensus.cc:614] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:06.156239   929 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:06.211864   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.213253   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.213949  4393 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:06.214237   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.153812   928 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:06.163496   926 raft_consensus.cc:2410] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:14:06.166765   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.177232   821 leader_election.cc:400] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:06.216837   821 leader_election.cc:403] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:14:06.177234   924 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:06.183282   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:06.184779   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:06.184969   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:06.218407   846 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.186453   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:06.219463   852 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:06.219923   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:06.189347   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:06.191427   849 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:06.141849  3803 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.163707  4337 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.177763  3926 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.203761   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.163370  4424 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.224182  4393 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.224713  4423 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.226421   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.252703   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.253494   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
W20260709 13:14:06.229560   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.231361   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:06.247666   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.280395   848 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.188499  4402 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:14:06.248104   922 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:06.286175   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.239928  4023 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.256435   849 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:06.304634   923 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:14:06.284315  1004 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:06.248567   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.305655   924 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.305652   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.308779  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:06.310483  1003 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:14:06.311726  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.312037   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:06.312387   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.312959   925 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:14:06.313089   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.313920   926 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:06.315990  3987 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.000s
I20260709 13:14:06.251931  4393 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.317052   924 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:06.318421   848 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:06.307847   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:06.208845  4336 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:06.322105  1004 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:06.278319  4434 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.216746  4359 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:06.236006  4437 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:14:06.338798  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:06.208525  4205 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.357239  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:06.326735  4394 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:06.327777  4439 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:06.338790  4407 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:06.387169   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:06.316377  3987 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:06.353850  4366 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.964s	user 0.011s	sys 0.004s
I20260709 13:14:06.316673  4023 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:06.402000   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.285907  4386 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.353129  4428 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:06.281592  4332 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:06.476758   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 89655.1us
I20260709 13:14:06.359588  4378 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.335072  4434 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.421119  4205 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:06.488490  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 131360us
I20260709 13:14:06.423089  4407 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.432864  4368 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:06.496367  1002 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:06.383581  4335 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:14:06.504868   853 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:06.484027  4255 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:06.422132  4123 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
W20260709 13:14:06.521908   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.522746   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.505486  4407 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.499908  4439 raft_consensus.cc:2764] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:14:06.423271  4394 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:06.556396  4424 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:06.557626  4441 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:14:06.528522  4437 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:06.536268  4332 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.560830   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1801) took 1353 ms. Trace:
I20260709 13:14:06.531231  4255 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:06.529203  4452 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:06.566390   850 rpcz_store.cc:276] 0709 13:14:05.207676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.211058 (+  3382us) service_pool.cc:224] Handling call
0709 13:14:05.228258 (+ 17200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.261531 (+ 33273us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:14:05.617581 (+356050us) log.cc:844] Serialized 14 byte log entry
0709 13:14:06.129509 (+511928us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.130164 (+   655us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.154584 (+ 24420us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:06.155096 (+   512us) log.cc:844] Serialized 14 byte log entry
0709 13:14:06.161028 (+  5932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.161423 (+   395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.560555 (+399132us) raft_consensus.cc:1685] finished
0709 13:14:06.560585 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.560755 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28117,"prepare.queue_time_us":484131,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":3169,"raft.queue_time_us":57270,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":53832,"spinlock_wait_cycles":5920128,"thread_start_us":751922,"threads_started":2,"wal-append.queue_time_us":513401}
I20260709 13:14:06.562163  4378 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.546139  4315 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.577555   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 400398us
I20260709 13:14:06.578617  4452 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:06.582921   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:06.592574  4439 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:06.593924   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:06.543368  4368 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:14:06.612716   821 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:06.629698   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 440402us
I20260709 13:14:06.621907   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:06.631829  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 124919us
I20260709 13:14:06.638412   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:06.639748   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:06.641270   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:06.584422  4336 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:06.625913  4438 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 13:14:06.616926  4439 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:06.578567  4378 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:06.586457  4315 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.597440  4424 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:14:06.674592   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:06.679546  4447 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:14:06.681100  4023 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:06.681660  4444 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
W20260709 13:14:06.683454  4455 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:14:06.684235  4446 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:14:06.681550  4367 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:06.637046  4359 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:06.639281  4437 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:06.689487   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:06.652321  4448 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:06.630577  4453 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:06.699232   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.739517   926 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:06.724649   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 504795us
I20260709 13:14:06.667692  4439 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.637605  4407 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.730891   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.733008  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:06.575018  4255 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.731817   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.732349   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:06.665280  4428 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:14:06.762558   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.772274   999 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:06.685964  4450 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:14:06.732677  4437 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.790277   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.796324   928 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:06.747866  4475 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.748242  4439 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:06.820363   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.790665  4448 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.832221   926 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:06.833359   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.834534   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.835098   926 raft_consensus.cc:2376] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:14:06.763465  4434 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:06.838317   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.802466  4437 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.861737   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1819) took 1040 ms. Trace:
I20260709 13:14:06.862854   995 rpcz_store.cc:276] 0709 13:14:05.797741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.797885 (+   144us) service_pool.cc:224] Handling call
0709 13:14:05.841072 (+ 43187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.989324 (+148252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.989368 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.989825 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.193979 (+204154us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:06.194032 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.194128 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.837440 (+643312us) raft_consensus.cc:1685] finished
0709 13:14:06.837459 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.837577 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60102,"raft.queue_time_us":84692,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":28303,"thread_start_us":165977,"threads_started":2,"wal-append.queue_time_us":213437}
I20260709 13:14:06.856956   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:06.828526  4442 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:06.858572  4492 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:06.859483  4428 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:06.859774  4450 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:06.888934  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.890352   926 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:06.914045   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.934621   926 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:06.936101   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:06.937065   926 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:06.938381   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:06.939627   846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ec15122641447a3bd48017eb990578d"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:06.940371   846 raft_consensus.cc:606] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:06.940434   926 raft_consensus.cc:2410] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:06.852787  4448 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.834033  4475 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:06.949218   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.115975   926 raft_consensus.cc:2376] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9. Re-sending same reply.
I20260709 13:14:06.956403   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:06.963924   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 147920us
I20260709 13:14:06.921916  4378 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:07.134119   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.050140   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 305175us
I20260709 13:14:07.123489   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.131330   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.138046   846 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:07.113863  4332 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.003s
I20260709 13:14:07.139326   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.135429   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.140331   847 raft_consensus.cc:2368] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:06.973343   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:06.977042  4478 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:14:06.976069  4476 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:14:07.160632   852 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:07.130784   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 473) took 1161 ms. Trace:
W20260709 13:14:06.977622  4467 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:14:07.163412   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.164636   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:06.977643  4402 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.975857  4451 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.928674  4428 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.214083  4451 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.216202  4428 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.216285  4451 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:07.216790  4445 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.466s	user 0.000s	sys 0.002s
I20260709 13:14:07.216816  4428 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:07.127591  4413 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.903s	user 0.008s	sys 0.007s
I20260709 13:14:07.163637   926 rpcz_store.cc:276] 0709 13:14:05.969729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.127312 (+1157583us) service_pool.cc:224] Handling call
0709 13:14:07.130312 (+  3000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.130490 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.130514 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.130556 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:07.130585 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.130635 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.130730 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:07.172787   929 raft_consensus.cc:2410] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:07.174039  4489 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:07.174491  4494 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:07.175519  4475 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:06.976935  4459 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.177088  4448 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.177254   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.177745  4498 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:06.936120  4492 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.178556  4402 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.179682  4496 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:07.180245  4307 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.180727  4332 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.183923  1001 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:14:07.185487  4403 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.186691   849 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:07.188517  4205 consensus_meta.cc:287] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.004s
W20260709 13:14:07.188946   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.190709  4473 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:07.271862  4496 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.191118  4337 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.244831  4475 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:14:07.278149  4337 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.280200  4337 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.192348   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2122) took 1005 ms. Trace:
I20260709 13:14:07.192767  4502 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:07.192938  4393 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.000s
I20260709 13:14:06.989426  4315 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.281865  4437 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.282565   924 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:07.195214  4439 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.283792   850 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:07.285450  4439 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.237723  4413 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.286078  4439 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:07.286593   922 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:07.197258  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1821) took 1357 ms. Trace:
I20260709 13:14:07.205886   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2126) took 1013 ms. Trace:
I20260709 13:14:07.208298  4394 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.209065   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 1903) took 2032 ms. Trace:
I20260709 13:14:07.210294  4356 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:06.972286  4407 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:06.990108  4452 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.220157   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:07.220630   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 476) took 1076 ms. Trace:
I20260709 13:14:07.221338   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.287878   854 rpcz_store.cc:276] 0709 13:14:06.192897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.216118 (+ 23221us) service_pool.cc:224] Handling call
0709 13:14:06.216440 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:06.350558 (+134118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.350602 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:06.379860 (+ 29258us) log.cc:844] Serialized 52 byte log entry
0709 13:14:06.580337 (+200477us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:06.580398 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.580503 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.205600 (+625097us) raft_consensus.cc:1685] finished
0709 13:14:07.205628 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.205815 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19432,"raft.queue_time_us":68764,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":13731,"thread_start_us":206133,"threads_started":2,"wal-append.queue_time_us":679204}
I20260709 13:14:07.299166   846 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:07.234555   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.235962   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.241222   926 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:07.302734   855 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
W20260709 13:14:07.243366   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66133.4us
I20260709 13:14:07.217814  4428 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.233425  4451 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.249733  4491 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.271654  4423 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.274610  4386 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.275318  4473 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:07.246841  4459 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.326195   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1893) took 1043 ms. Trace:
I20260709 13:14:07.284955  4492 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.286563  4439 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.192080  4468 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.288089   923 rpcz_store.cc:276] 0709 13:14:05.176756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.176971 (+   215us) service_pool.cc:224] Handling call
0709 13:14:05.177328 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.308365 (+1131037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.208788 (+900423us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400080a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.208997 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2591650560}
I20260709 13:14:07.281412   853 rpcz_store.cc:276] 0709 13:14:06.187189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.187323 (+   134us) service_pool.cc:224] Handling call
0709 13:14:06.201219 (+ 13896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.507912 (+306693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.192036 (+684124us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54003b0bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.192266 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697740672}
I20260709 13:14:07.287756  1002 rpcz_store.cc:276] 0709 13:14:05.839569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.859721 (+ 20152us) service_pool.cc:224] Handling call
0709 13:14:05.909628 (+ 49907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.502953 (+593325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.196942 (+693989us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54001428d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.197185 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348587904}
I20260709 13:14:07.250703  4498 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:07.297120  4443 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.917s	user 0.003s	sys 0.002s
I20260709 13:14:07.313329  4394 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.298009   847 raft_consensus.cc:606] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:07.332378   847 raft_consensus.cc:614] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:07.305056  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.297967   927 rpcz_store.cc:276] 0709 13:14:06.144642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.216667 (+1072025us) service_pool.cc:224] Handling call
0709 13:14:07.217011 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.217215 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.217240 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.220354 (+  3114us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:07.220400 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.220461 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.220576 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:07.332937   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 159111us
I20260709 13:14:07.306608   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.287577  4413 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.288141  4448 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:07.277251  4496 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.326763   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1827) took 1356 ms. Trace:
I20260709 13:14:07.241917  4494 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.326599   922 rpcz_store.cc:276] 0709 13:14:06.282722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.285934 (+  3212us) service_pool.cc:224] Handling call
0709 13:14:07.325985 (+1040051us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54001699d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:07.326130 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2298985344}
W20260709 13:14:07.290002  3926 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.023s	user 0.013s	sys 0.000s
I20260709 13:14:07.504575   999 rpcz_store.cc:276] 0709 13:14:05.970016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.970144 (+   128us) service_pool.cc:224] Handling call
0709 13:14:06.141390 (+171246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.772847 (+631457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.326516 (+553669us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540034eed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.326706 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450418432}
I20260709 13:14:07.505370  3926 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.506218  4494 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.506531   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:07.507275  4497 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:07.507512  3926 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.507894   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.495983  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 428) took 1213 ms. Trace:
I20260709 13:14:07.496258  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.496671   928 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:07.324618  4423 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.317986  4451 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.323879  4386 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:07.555506   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 222432us
I20260709 13:14:07.317278  4452 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:07.744632   967 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 1.887s (SENT) [suppressed 21 similar messages]
I20260709 13:14:07.744691  4452 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.747102  4452 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.747162   846 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:07.510079  4448 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.510706   923 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:07.748108   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1903) took 1215 ms. Trace:
I20260709 13:14:07.748317   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.748436   923 rpcz_store.cc:276] 0709 13:14:06.532850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.352621 (+819771us) service_pool.cc:224] Handling call
0709 13:14:07.510393 (+157772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.747883 (+237490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.748058 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:07.508697  4255 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:14:07.749202  4448 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.749555   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.749879  4255 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:07.750180   846 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 4.
I20260709 13:14:07.750288  4255 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:07.514802   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:07.752584   850 raft_consensus.cc:606] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.752904   850 raft_consensus.cc:614] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:07.508808  1001 rpcz_store.cc:276] 0709 13:14:06.283036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.283593 (+   557us) service_pool.cc:224] Handling call
0709 13:14:06.289039 (+  5446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.269193 (+980154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.359513 (+ 90320us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b5400047ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.495923 (+136410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057358208}
I20260709 13:14:07.527861  4437 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.503305  4394 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.754290   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 474) took 1759 ms. Trace:
I20260709 13:14:07.560515   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 478) took 1039 ms. Trace:
I20260709 13:14:07.755641  4394 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:07.560421   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 253882us
I20260709 13:14:07.755295   927 rpcz_store.cc:276] 0709 13:14:06.521226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.344742 (+823516us) service_pool.cc:224] Handling call
0709 13:14:07.560008 (+215266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.560208 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.560232 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.560270 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:07.560298 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.560347 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.560457 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:07.756662   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.562280  4402 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.528492  4473 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.562283  4507 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:07.562042  4286 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.562834  4491 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.757493  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:07.565062   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.757822  4507 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:07.757875  4286 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.515873  4501 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.759174  4501 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.758932  4507 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.759593  4501 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.761988   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.565330   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1886) took 1489 ms. Trace:
I20260709 13:14:07.762997   929 rpcz_store.cc:276] 0709 13:14:06.075861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.214030 (+138169us) service_pool.cc:224] Handling call
0709 13:14:07.565125 (+1351095us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:07.565253 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2202146688}
I20260709 13:14:07.756120 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:07.763667   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.764216   922 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:14:07.567210   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 59382.6us
W20260709 13:14:07.568961  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53816.4us
W20260709 13:14:07.601897   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.766654   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.566710  4496 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.567219  4364 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.768443  4329 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:07.768494  4364 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.769006  4329 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.769109  4364 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.769447  4329 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.770180  4473 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.770834  4501 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.564088  4498 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.771381  4491 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.771754  4045 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.771941  4498 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:07.772393  4498 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.772277  4045 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.772837  4498 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.631289  4439 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:07.640974   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.774392  4045 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.628916  4423 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.654736  1601 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:07.652906   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 475) took 1656 ms. Trace:
I20260709 13:14:07.775380   921 rpcz_store.cc:276] 0709 13:14:05.996795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.184943 (+1188148us) service_pool.cc:224] Handling call
0709 13:14:07.185321 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.506846 (+321525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.506874 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.506914 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:07.506942 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.652695 (+145753us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400328cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.652740 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.652854 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738970496}
I20260709 13:14:07.644946  4428 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.777078  4142 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.648165  4451 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.683425  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:07.683077  4424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.014s	sys 0.028s
I20260709 13:14:07.777550  4142 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:07.778155  4424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.014s	sys 0.028s
I20260709 13:14:07.778599   999 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:07.778867   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.683938  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.779693   997 raft_consensus.cc:2410] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:07.780023  1593 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:07.685431  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.780616   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:07.686765  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.781165   995 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.653985  4335 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.781831  4335 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.782981  4394 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:07.682745  4459 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.783705  4451 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:07.784425  4451 consensus_queue.cc:1243] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:07.784878  4451 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:07.704506  4326 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:07.785171  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.785360  4326 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:07.786350  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:07.786275  4326 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.702929  4386 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:07.721167   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1856) took 2116 ms. Trace:
I20260709 13:14:07.787948   930 rpcz_store.cc:276] 0709 13:14:05.605078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.645356 (+ 40278us) service_pool.cc:224] Handling call
0709 13:14:05.896846 (+251490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.038152 (+141306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.697110 (+1658958us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b5400168fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.720771 (+ 23661us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400168fd0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:07.721016 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672397440}
I20260709 13:14:07.788515  4013 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:07.789266  4013 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:07.789327  4482 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:07.789767  4482 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:07.790652  4482 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.791847   999 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:07.791944  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:07.792747  1004 raft_consensus.cc:2368] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:07.792961  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.726679  3987 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:07.730129   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 477) took 1347 ms. Trace:
I20260709 13:14:07.793798  1002 raft_consensus.cc:2410] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:07.787411  4326 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.738202   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:07.794427   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.795012   849 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:07.738890   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.793658   926 rpcz_store.cc:276] 0709 13:14:06.382520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.311490 (+928970us) service_pool.cc:224] Handling call
0709 13:14:07.311909 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.565317 (+253408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.565346 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.565802 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.641877 (+ 76075us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:07.641936 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.729705 (+ 87769us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400090454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.729759 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.729810 (+    51us) raft_consensus.cc:1685] finished
0709 13:14:07.729832 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.730062 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1898,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":10723,"spinlock_wait_cycles":581963008,"thread_start_us":75628,"threads_started":1,"wal-append.queue_time_us":116698}
I20260709 13:14:07.795909   852 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:07.795849  4364 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.739728   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 1905) took 1176 ms. Trace:
I20260709 13:14:07.796676   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.796824   924 rpcz_store.cc:276] 0709 13:14:06.563174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.717598 (+154424us) service_pool.cc:224] Handling call
0709 13:14:07.739529 (+1021931us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54003cbc54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:07.739676 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224507136}
I20260709 13:14:07.740688   853 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:14:07.797716  4450 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:07.740767   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:07.777813  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.743062  4514 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.798496   851 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.762609   927 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:07.320143  4504 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:07.799243   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:07.701522  4453 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.000s	sys 0.024s
I20260709 13:14:07.756711  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.802520   999 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:07.802788  1598 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
W20260709 13:14:07.802515  4453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.000s	sys 0.025s
I20260709 13:14:07.791963   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.803798  4453 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:07.804397  4246 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.798070  4498 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.798480  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.804863   895 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:07.765645 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:07.798103  4450 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:07.806031  4450 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.804873  4364 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:07.808048  4498 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.754652   925 rpcz_store.cc:276] 0709 13:14:05.995302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.131591 (+1136289us) service_pool.cc:224] Handling call
0709 13:14:07.171979 (+ 40388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.313789 (+141810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.313816 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.313855 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:07.313897 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.754091 (+440194us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54004448d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:07.754138 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.754241 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23340,"raft.queue_time_us":23751,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6985,"spinlock_wait_cycles":271419776}
I20260709 13:14:07.799721  4504 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.811638  4335 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.813439  4504 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.800859   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.801581  4394 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:07.814425  4394 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:07.751135  4255 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.815490  4394 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.816617  4142 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:07.817461  4337 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:07.817893  4337 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.818408  4337 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.819105  4255 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.820008  1000 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:07.785218  4451 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:07.823784   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:07.824709   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.804602  4453 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.798925  4514 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.825465  4498 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:07.825870  4498 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.825980  4452 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:07.826274  4498 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.826427  4452 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.826903  4452 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:07.826787   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.827042  4514 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.827503   853 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:07.827741  4514 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:07.742991   847 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:07.740223   855 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:07.828891  4482 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:07.824302  4451 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.830504  4451 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:07.832366   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 48404.2us
I20260709 13:14:07.833268  4246 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.834491   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:07.835966  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.838465  1004 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:14:07.838953   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.836701   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.839687  4393 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:07.828238   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:07.840341  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 49533.4us
I20260709 13:14:07.840689  4393 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:07.840870  4514 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.841104  4393 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:07.841872  4326 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:07.842412  4326 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:07.842036  4393 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.843143   995 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:07.843289  4326 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.843717  4393 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.846213  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.851135  4336 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:07.837601  4246 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.855402  4246 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.842306   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:07.864745  4286 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.865027   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:07.866894  4457 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.867569  4457 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.870126  4457 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:07.871019   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.871799  4501 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:07.873140  4514 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.873678  4514 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.874125  4514 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.874475  4246 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.877146  4246 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.877928  4307 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:07.880609  4307 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:07.881047  4307 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.884169  4246 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.886360   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:07.921083   855 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.922499  4452 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.887373   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.888702   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.924046   927 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:07.924889   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.925999   925 raft_consensus.cc:2376] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12. Re-sending same reply.
I20260709 13:14:07.888972   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.889261   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.927201   924 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:07.890266   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.891335   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.892372   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.892838   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.929804   922 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:07.892323  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.931650  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.932233  1600 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:07.894590  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.895818  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.897845  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.898983   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.899819  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.901320  1596 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:07.901221  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.903949  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:07.936192  1595 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:07.905205   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.938128  4246 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.931936  4452 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.941954   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.943681   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.943903  4450 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.946238  4450 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:07.914093   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:07.916821  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.917241   819 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.918526  4498 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.992070   852 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:07.992290   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 75140us
I20260709 13:14:07.993706  4498 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:07.993888   848 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:07.875103  4517 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:07.889561   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:07.959434   896 leader_election.cc:400] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:07.995738  4517 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:07.995949  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:07.996930  4517 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.933687   855 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:07.816946  4394 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.944428   929 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:07.912396  4329 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:07.947955   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.008249   930 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.009186  4255 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.009716  4255 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:07.950965   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.011862  4255 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.011916   995 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:14:07.907302   851 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:07.953697   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:07.955484   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.014942   928 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:07.958348   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:08.016278  4255 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:08.016798  4255 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.017235  4255 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:08.015980   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 57735.8us
I20260709 13:14:08.021641   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:08.017935  4329 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:07.960085   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 123134us
I20260709 13:14:08.022382  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:08.025660   895 leader_election.cc:400] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:08.025938   895 leader_election.cc:403] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:08.024343  4407 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:08.026475  4407 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:08.026988  4407 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.960579  4326 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:08.029788  4498 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.029912   895 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.030483  4498 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.031100  4393 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:07.961795   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.032475   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.032723  4393 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:08.033800  4498 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:07.968102   967 connection.cc:256] RPC call timeout handler was delayed by -1.6821339420000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 13:14:08.035808   967 connection.cc:256] RPC call timeout handler was delayed by -1.6758339420000539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:14:08.036158   967 connection.cc:256] RPC call timeout handler was delayed by -1.5624339420000979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:14:08.036423   967 connection.cc:256] RPC call timeout handler was delayed by -1.4526339419999204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:14:08.036688   967 connection.cc:256] RPC call timeout handler was delayed by -1.417533942000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:14:08.036934   967 connection.cc:256] RPC call timeout handler was delayed by -1.3896339420000459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 13:14:08.037194   967 connection.cc:256] RPC call timeout handler was delayed by -1.3041339419999076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 13:14:08.037223   851 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:08.037546   967 connection.cc:256] RPC call timeout handler was delayed by -1.2699339420000797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 13:14:07.970674   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.974660  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:08.038352   854 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:07.978585  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:07.972683  4337 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.040366   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.040556  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.041121   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:07.983134  4307 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.041693  1566 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:07.979308  4246 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:14:08.042325  1598 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:07.993185  4453 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.014897   924 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.042632   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.027295  4494 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:08.044214  4336 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:08.044706  4336 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:08.045506  4336 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.046583   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.033947  4326 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:08.047772  4326 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:08.048655  4326 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.049614  4326 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.022091  1596 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:14:08.042796  4453 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.050264  4453 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.042106  4337 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.052420  1592 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:08.012514  4452 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.053119  4452 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.055176   852 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:07.955389  4458 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:14:08.055765  4286 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:08.056376  4501 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:08.056751  4501 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:08.056843   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.057857  4453 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.058660   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:08.059475  4337 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:08.060034  4337 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:08.023244   930 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:08.060371  4453 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.061182  4337 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.067759  4420 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:08.068380   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.069211   852 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:08.041711   855 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:08.069757  4423 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.070880  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:08.067761  4498 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.058702  4501 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.070456  4423 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.072805  4501 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.073972  4423 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:08.071875  4498 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.060212  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:08.077304   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:07.995738   896 leader_election.cc:403] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:08.078109   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:08.056018   924 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:08.055240  4452 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.081274  4319 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:08.083510  4356 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:08.084649  4142 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:08.084986  4142 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:08.085763  4142 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:08.086898  4535 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:08.090260  4286 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.090756  4286 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:08.092584   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.095188   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.100262   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.110654   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.110785   855 raft_consensus.cc:2368] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:08.064664   969 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:08.079969  4458 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:08.079933  4453 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:08.112860   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 48265.4us
I20260709 13:14:08.101804   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.102877   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.104046   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.114430   851 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:08.104809   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:08.108264   967 connection.cc:256] RPC call timeout handler was delayed by -1.1673339419999138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 13:14:08.109714   853 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:14:08.098280   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:08.112026   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 152729us
I20260709 13:14:08.115681   853 pending_rounds.cc:85] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 4
I20260709 13:14:08.116055   853 pending_rounds.cc:107] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:14:08.116366   853 raft_consensus.cc:2889] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:08.116598   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:08.115895   967 connection.cc:256] RPC call timeout handler was delayed by -1.1106339420000495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:14:08.117353   967 connection.cc:256] RPC call timeout handler was delayed by -1.0278339419999156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
I20260709 13:14:08.119549   846 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:07.958179  4514 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.123387   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.125293   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:08.136905   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.133692   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.134050   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.137516   929 raft_consensus.cc:2376] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5. Re-sending same reply.
I20260709 13:14:08.137725   928 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:08.134891   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.138592   924 raft_consensus.cc:2376] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10. Re-sending same reply.
I20260709 13:14:08.139010   852 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:08.135435   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.067790  4450 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.135052   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.091765  4307 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.069619  4420 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:08.140908   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:08.276407   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.276960   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 136126us
I20260709 13:14:08.140748   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:08.142380   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.143855   929 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:08.111163  4452 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:08.111626  4482 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.279152   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.279302   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.148965   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.149561   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.280484   928 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:08.280575   924 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.281591   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.282205   922 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:08.149654   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.119262  4453 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.147706   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:08.150800   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.131323  4458 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:08.156599  4498 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.160207   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.288966   925 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:14:08.167201   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.167716   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.290151   930 raft_consensus.cc:2376] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11. Re-sending same reply.
I20260709 13:14:08.146567  4386 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:14:08.151356  3987 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:14:08.148123  4337 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.146992  4326 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.146911  4319 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:14:08.164077  4246 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.132731  4517 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.128073  4415 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.127991  4496 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:08.139853  4494 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.111167  4255 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.344341  4494 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:08.263052   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131717us
I20260709 13:14:08.122035  4394 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.346222  4494 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.289513  4420 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.346654  4394 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.348858   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.349370  4394 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.277859   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.350020   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:08.350420   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 72644.3us
I20260709 13:14:08.351163   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.352455   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.353440   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:08.353832  4319 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.354578  4393 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:08.355168  4326 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.355436  4393 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.346225  4255 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.356297  4393 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.357291  4393 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.357906  4393 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.335920  4415 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.360492  4415 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:08.361135  4529 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:14:08.354530  4319 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.363955   895 leader_election.cc:400] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:08.351296   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.365032   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:08.365918   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.283025   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.284585   923 raft_consensus.cc:2376] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7. Re-sending same reply.
I20260709 13:14:08.367594   927 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:08.368296   922 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:08.290436  1598 raft_consensus.cc:2955] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 3 to 7, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.279831   929 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:08.291229   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:08.291277   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.291931   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 151291us
I20260709 13:14:08.299468   924 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:08.372781   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:08.303684  4329 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:08.385891  4329 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:08.386575  4329 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.286873  4491 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:14:08.279814  4142 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.313263   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.320816   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:08.395955   921 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:08.318228   749 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 11, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.395937  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:08.398741   749 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 6, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.399736   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.400290   749 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.401827   749 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 7, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.286653  4473 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.286614  4453 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.326357   851 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:08.396385   925 raft_consensus.cc:2376] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9. Re-sending same reply.
I20260709 13:14:08.270601  4450 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.312184  4542 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:08.336158  4517 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.342218  4458 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.286188   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.311405  4452 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:08.380174  4423 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:08.382210  4286 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.340353  4246 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.385237  4420 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.405807   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.407435   896 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.403194   749 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.289204  4407 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.409951  4056 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:08.420625  4056 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:08.322114  4337 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.419299   749 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.405359  4494 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:08.409933  3987 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.435632   749 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:08.437129   928 raft_consensus.cc:2410] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:08.410077  4457 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:08.434069   751 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 231) took 17678 ms (client timeout 15000 ms). Trace:
I20260709 13:14:08.414244  4496 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:08.437830   749 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 6, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 6 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.415132  4517 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.425793  4056 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:08.452490  4056 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.421849  4502 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:08.456071  4502 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:08.461901  4286 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.462805  4452 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.464056   749 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.469308  4450 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.468974   749 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.445588  1595 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:08.466904  4517 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.466398  4494 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.430656  4337 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.474669   926 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:08.472200   749 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 8, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 8 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.436378  4420 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.473548  4255 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:08.443192  4473 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.426592  4491 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:14:08.421031  4394 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:14:08.500388   749 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:08.442269   751 rpcz_store.cc:269] 0709 13:13:50.751308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.751665 (+   357us) service_pool.cc:224] Handling call
0709 13:14:08.429249 (+17677584us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:06.001512 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:06.098493 (+ 96981us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:06.098584 (+    91us) write_op.cc:435] Acquired schema lock
0709 13:14:06.098618 (+    34us) tablet.cc:662] Decoding operations
0709 13:14:06.099778 (+  1160us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:06.099885 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:06.099915 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:14:06.112593 (+ 12678us) tablet.cc:701] Row locks acquired
0709 13:14:06.112647 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:14:06.117513 (+  4866us) write_op.cc:270] Start()
0709 13:14:06.117703 (+   190us) write_op.cc:276] Timestamp: P: 1783602846112764 usec, L: 0
0709 13:14:06.117725 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:06.118406 (+   681us) log.cc:844] Serialized 3614 byte log entry
0709 13:14:07.195479 (+1077073us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:07.697434 (+501955us) op_driver.cc:464] REPLICATION: finished
0709 13:14:07.857215 (+159781us) write_op.cc:301] APPLY: starting
0709 13:14:07.857297 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:07.857555 (+   258us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:07.857581 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:07.858771 (+  1190us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:07.859033 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:14:07.859082 (+    49us) write_op.cc:312] APPLY: finished
0709 13:14:07.859385 (+   303us) log.cc:844] Serialized 71 byte log entry
0709 13:14:07.860382 (+   997us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:07.860522 (+   140us) write_op.cc:454] Released schema lock
0709 13:14:07.860619 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:07.923896 (+ 63277us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:07.923928 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":109360,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":159044,"apply.run_cpu_time_us":3000,"apply.run_wall_time_us":66651,"num_ops":9,"prepare.queue_time_us":221170,"prepare.run_cpu_time_us":46993,"prepare.run_wall_time_us":217398,"raft.queue_time_us":58303,"raft.run_cpu_time_us":24095,"raft.run_wall_time_us":230399,"replication_time_us":1578842,"spinlock_wait_cycles":49877888,"thread_start_us":447286,"threads_started":4,"wal-append.queue_time_us":198112}]]}
I20260709 13:14:08.487932  4494 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.521953   928 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:08.522473   928 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:08.514405   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 366713us
I20260709 13:14:08.485523  4514 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.501845   749 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.551079  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.552772  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.549817   749 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 8, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 8 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.553418  1004 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:08.554504   749 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 7, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:08.556417   749 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.558499   749 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 6, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.523367   928 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.563706   928 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.564049   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.592682   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.600006  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:08.492988  4415 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.607316  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:08.608220  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:08.608983  1002 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:08.611057   851 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.566677   749 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 8, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3), now has a pending config: VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1), VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3), VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 8 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.500655  4255 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:08.510454  4473 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.614395  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:08.617611  1595 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:08.618247   749 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 9, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:08.628870  1592 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:08.616083   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 197436us
I20260709 13:14:08.633646   851 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:14:08.630674   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.625921   749 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 8, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 8 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.643309   749 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 4, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 4 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:08.621549  4246 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.647501   853 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.649828   851 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.650218   749 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 8, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 8 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.655665  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:08.656690   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.646004   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.655565   749 catalog_manager.cc:5697] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.663470   749 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 5, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 5 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:08.666747   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.650362  3987 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:08.662933  4542 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:08.667760   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 252168us
I20260709 13:14:08.669332   967 leader_election.cc:400] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
W20260709 13:14:08.674036   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 266660us
I20260709 13:14:08.689359  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.657738  4407 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.695557  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.689433   967 leader_election.cc:403] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:08.697593   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.698635  4458 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.698750  4548 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.699251   896 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.699321  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.702591   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.653012  4545 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:14:08.714879   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.720523   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51314.8us
I20260709 13:14:08.720834   849 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:08.721333   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:08.723901   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65045.8us
I20260709 13:14:08.726490   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.729213   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.731163   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.732010   855 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:08.673223  4547 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:08.711480  4246 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.740185   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.711709  4496 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.738232   895 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:08.573752  4451 consensus_queue.cc:1175] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "597687e464174d1ea6043161e037d240" responder_term: 6 status { last_received { term: 6 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:14:08.745553  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46292.7us
I20260709 13:14:08.733129  4458 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.734483  4501 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.697259  4056 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:08.744184   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:08.736991  4517 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:08.756692   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57515.1us
I20260709 13:14:08.741514  4407 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:08.758433   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.742610  4491 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:08.761529   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:08.760545  4452 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:08.749591  4496 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.766793  4356 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:08.784734  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:08.765297  4407 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:08.768182  4550 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.771255  4517 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:08.756428  4501 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:08.785797   821 leader_election.cc:336] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.562s (SENT)
I20260709 13:14:08.792807   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.780032  4246 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:08.798763  4517 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:08.799774  4517 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:08.802246  4553 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:08.753062  4542 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.784438  4255 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:08.804875   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 60766.2us
I20260709 13:14:08.806417 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:08.767489  4329 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:08.813158   821 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.572s (SENT)
I20260709 13:14:08.815769   847 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.826583   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.794971  4473 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.795708  4246 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.828760   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:08.839102   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:08.831095  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:08.831367   849 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:08.842077   999 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:08.843880  1000 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:08.839399   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.850466  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:08.809356  4507 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:08.854044   847 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:08.851250  1003 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:08.859320   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 119226us
I20260709 13:14:08.859546   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:08.864276   922 raft_consensus.cc:2410] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:14:08.815106  4450 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.858495   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.832404  4420 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:08.843883  4246 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.866948   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.831044  4498 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:14:08.869647  4356 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.889271   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.869535  4547 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:14:08.874238  4329 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:08.888809   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:08.869450  4450 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:08.890551  4393 consensus_meta.cc:287] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.003s
I20260709 13:14:08.932286   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:08.888471  4319 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.876003  4564 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:14:08.913283  3987 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:14:08.874879  4407 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.944618  4356 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:08.947943  4566 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:14:08.942776  4550 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:08.950980  4502 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.959383  4573 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:08.960033  4496 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.961791  4502 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:08.965575  4561 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:14:08.972028  4550 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.962456  4319 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:08.976565  4319 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:08.944029  4393 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.977612  4319 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:08.979241  4319 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:08.972818  4570 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:08.980072  4393 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:08.982856  4550 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:08.963672  4563 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:14:08.996944  4453 consensus_meta.cc:287] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.581s	user 0.008s	sys 0.002s
W20260709 13:14:08.981783  4142 consensus_meta.cc:287] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 13:14:09.003192  4482 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.007768  4507 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.008406  4507 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.008338  4482 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.011034  4507 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.003741  4473 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.011793   995 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:09.012113  4473 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.015079  4473 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:09.016112  4337 consensus_meta.cc:287] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.003s
I20260709 13:14:09.016590  4337 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.004628  4517 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.018754  1592 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:14:09.018587  4517 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.019200  4450 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:09.021253   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 89125.1us
W20260709 13:14:09.024504   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 92347.1us
I20260709 13:14:09.034960   852 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:09.011466  4482 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.034407  4547 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:09.048236   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.060071  4573 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:09.057607  4420 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.022464  4453 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.033735  4580 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:09.076284   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.033134  4507 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:14:09.082906   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.083874   926 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.085812   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.086225   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.086374   927 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:09.086676   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.086766   923 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.085927   926 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:09.088044   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.089282   930 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:09.090219   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.090749   930 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.053664  4547 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:09.087134   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.100600   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.087224   921 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.102723  4393 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.098258   818 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.101909   924 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.103917   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.105026   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.105803   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.088406   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.107407   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.102821   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.108263   921 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:09.100636   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.050832  3987 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:14:09.050993  4142 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:14:09.111743  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.110666   926 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:09.052999  4564 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:14:09.136197  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.142143   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:09.145582   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47826.6us
I20260709 13:14:09.153259   924 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.151669   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.161293   930 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.168337   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.170008   930 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:09.171335   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.171833   930 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:09.173482   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.175143  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.175210   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.175899  1002 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.112056  4423 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.111676  4450 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:09.113188  4473 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:09.174348   921 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:09.174322   930 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
W20260709 13:14:09.164286  4494 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.591s	user 0.014s	sys 0.000s
I20260709 13:14:09.136485  4498 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:09.182163   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.183842   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:09.184268   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 125542us
I20260709 13:14:09.187865   926 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:09.190945   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.162989  4580 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:09.171829  4507 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:09.206427   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.208570   930 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:09.123201  4482 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:14:09.209105   930 pending_rounds.cc:85] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 4
I20260709 13:14:09.210896   930 pending_rounds.cc:107] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:09.211374 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:09.211352   930 raft_consensus.cc:2889] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.213930   930 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:14:09.215662   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.228376  1002 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:09.186515  4572 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:09.174589  4579 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:14:09.232321   926 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:09.184535  4473 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:09.181608  4450 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:09.186612  4498 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:09.204309  4494 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.208284  4517 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.209179  4542 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.213132  4423 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.241003  4572 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:09.276281   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85459.4us
I20260709 13:14:09.272414  4498 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.246573  4337 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.247682  4473 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:09.271816  4450 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.335974   926 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:09.241554  4393 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.337849   929 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 6: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:09.332923   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.358407   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.333253  4572 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.370239   923 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.380797  4393 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:09.412039  4575 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:14:09.440946   927 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:09.204905  4547 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.405663  4580 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:09.445222   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 86896.7us
I20260709 13:14:09.498579   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.500484   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:09.357089  4561 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:14:09.490276  4246 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
W20260709 13:14:09.493422  4584 log.cc:927] Time spent T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.249s	user 0.004s	sys 0.000s
I20260709 13:14:09.515328  1002 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:09.441798  4142 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.523916  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.449826  4546 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:09.505600  4586 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.228s	user 0.000s	sys 0.005s
W20260709 13:14:09.504866  4587 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.260s	user 0.001s	sys 0.002s
I20260709 13:14:09.493796  4453 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.492928  4517 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:09.543678   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.549600   998 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:09.553635   848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:09.554239   848 raft_consensus.cc:606] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.554646   848 raft_consensus.cc:614] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:09.540268  4407 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:09.550704  4565 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.519531  4246 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.552273  4517 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.575179  4337 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:09.577940  4565 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.578161  4337 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.579433  4517 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:09.580120  4565 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.580267  4337 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:09.576442   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.539686  4142 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.589648   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.590965  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:09.563418  4407 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.574991  4594 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:09.600359   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.598206   921 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:09.570042  4592 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:09.545805  4546 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.601146   998 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:09.603454   848 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:09.608860   846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ec15122641447a3bd48017eb990578d"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:09.609467   846 raft_consensus.cc:606] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:09.609822   846 raft_consensus.cc:614] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:09.610365   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.607120  1002 raft_consensus.cc:2368] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:09.613901   998 raft_consensus.cc:2368] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:09.615378  1592 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:14:09.598816  4407 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:09.603991  4592 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.598248  4579 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:14:09.629143   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.623476  4494 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.639540   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.630539  4319 consensus_meta.cc:287] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.000s
I20260709 13:14:09.607620  4594 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.642146   998 raft_consensus.cc:2368] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:09.679770   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.680332   998 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:09.680474  4594 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:09.638440  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:09.645780   848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155790402004e4f930983841dfd8c64"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:09.682386   848 raft_consensus.cc:606] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:09.682729   848 raft_consensus.cc:614] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:09.646807  4319 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:09.684442 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:09.657243   928 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:14:09.684954   928 pending_rounds.cc:85] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 7
I20260709 13:14:09.685212   928 pending_rounds.cc:107] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:14:09.685447   928 raft_consensus.cc:2889] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.686120   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 496) took 1165 ms. Trace:
I20260709 13:14:09.659812  4393 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.665263   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.626756  4599 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:14:09.686405   928 rpcz_store.cc:276] 0709 13:14:08.521145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.521460 (+   315us) service_pool.cc:224] Handling call
0709 13:14:08.521814 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.656754 (+1134940us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540011a920. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b41d87 00007f0d02b42470 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c
0709 13:14:09.685873 (+ 29119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:09.686070 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380320512}
I20260709 13:14:09.667490  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.692166  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.693154  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.694299  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.695340  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.696086   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.669143   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.671737  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.698055  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.672894  4415 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:09.673537  4246 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.675024  4546 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:09.676990  4142 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:09.629889  4592 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:09.696478  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.689227   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.707487  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:09.708751 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:14:09.719298   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.727182   998 raft_consensus.cc:2410] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:09.728509  1003 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:09.729861  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.731966  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.708181  4494 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.734607   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.750995  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.750097  4494 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.752310  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.753363  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.753647  4494 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:09.754364  1000 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:09.760730  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.763772  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.758409  4578 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.804s	user 0.004s	sys 0.000s
I20260709 13:14:09.768397  1003 raft_consensus.cc:2410] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:14:09.724493  4473 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.774379   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.727816  4450 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:09.778434   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 165995us
I20260709 13:14:09.779807  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.782280   854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1820) took 1678 ms (client timeout 2016 ms). Trace:
W20260709 13:14:09.782681   854 rpcz_store.cc:269] 0709 13:14:08.100358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.101143 (+   785us) service_pool.cc:224] Handling call
0709 13:14:09.778573 (+1677430us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540019e6d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:09.778709 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316499200}
I20260709 13:14:09.783963   848 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:09.784190   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:09.773619  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.787129  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.791725   850 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:09.793030  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.793681  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.799979   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 110896us
I20260709 13:14:09.801628  4514 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.802179  4514 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.803933  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.805218  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.806303  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.806429  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:09.807006  1000 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:09.808579  4514 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:09.809849  4514 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:09.810446  4514 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.813071   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.814059   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.815114   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.815783  4594 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:09.816218  4594 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.816596  4594 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:09.787739  4415 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.820933   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.825183   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.825729   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33108 (request call id 439) took 1275 ms. Trace:
I20260709 13:14:09.826181   995 rpcz_store.cc:276] 0709 13:14:08.550211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.550327 (+   116us) service_pool.cc:224] Handling call
0709 13:14:09.825548 (+1275221us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54003e3854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:09.825677 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961615104}
I20260709 13:14:09.827821   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.827957   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.829537   928 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:09.830686   928 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:14:09.830776   923 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:09.831862   930 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:09.832859   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.833961   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.825822   925 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:09.837420   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:09.837657   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.839533   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:09.840269   852 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:09.839154   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.841531   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:09.842334   853 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:09.839213  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.844880   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:09.794709  4546 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.840714   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:09.851732   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:09.846485   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:09.852398   847 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:09.852627   852 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:09.852906   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.853499  1000 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:09.816428  4576 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.862s	user 0.001s	sys 0.000s
I20260709 13:14:09.861779   895 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.861601   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:09.862637   847 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.850883   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:09.864429   927 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.855400  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.856942   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.849025   928 raft_consensus.cc:1240] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:09.857735   926 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:09.845487   853 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:09.822857  4599 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:09.859959  4246 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:09.875142   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:09.789757  4142 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:14:09.864336   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:09.865414   821 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.680s (SENT)
I20260709 13:14:09.859007  4551 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:09.797039  4597 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 13:14:09.838282  4502 consensus_meta.cc:287] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.817s	user 0.013s	sys 0.002s
I20260709 13:14:09.870195  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.876543   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.879354   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.837524  4453 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:09.844292  4337 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:09.851351  4517 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:09.885310   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1856) took 1061 ms. Trace:
I20260709 13:14:09.886929   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:09.896446   927 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:09.850893  4579 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.900312   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:09.913779   995 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:09.874161  4580 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:09.914754   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:09.898898  1000 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:09.877276  4565 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:09.922207   852 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:09.905624   997 rpcz_store.cc:276] 0709 13:14:08.823959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.824181 (+   222us) service_pool.cc:224] Handling call
0709 13:14:08.850677 (+ 26496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:08.862149 (+ 11472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.862184 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:08.868072 (+  5888us) log.cc:844] Serialized 52 byte log entry
0709 13:14:08.948975 (+ 80903us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:08.949015 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.949096 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:09.757574 (+808478us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:09.874838 (+117264us) raft_consensus.cc:1685] finished
0709 13:14:09.874870 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:09.885214 (+ 10344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14006,"raft.queue_time_us":7236,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7659,"spinlock_wait_cycles":7684992,"thread_start_us":36493,"threads_started":1,"wal-append.queue_time_us":71397}
I20260709 13:14:09.926875   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:09.923374   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 84287.2us
I20260709 13:14:09.935461   847 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:09.938800   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:09.846027  4415 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.940490   847 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:14:09.861075  4451 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:14:09.946098  1000 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:14:09.952773   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 65938.8us
I20260709 13:14:09.869074  4546 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:09.905565  4517 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:09.962728  4572 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.905851  4337 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:09.945047  4575 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:14:09.965476  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 104907us
I20260709 13:14:09.931948  4393 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:09.968941  1003 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:09.962167  3987 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:09.977079   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.963872  4565 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:09.997828   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 133548us
I20260709 13:14:09.968127  4517 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.964465  4615 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:09.970641  4337 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.929960  4579 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:09.963390  4286 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 13:14:10.051252  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 101194us
I20260709 13:14:10.008749  4565 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.012017  4514 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:10.051409   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 112670us
W20260709 13:14:10.061323   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 195906us
I20260709 13:14:09.972131  4142 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:14:10.114346   995 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:10.058290  1000 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:09.976323  4542 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:14:10.127187  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.045441  4615 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.128499  4514 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:10.128981  4514 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.997170  4598 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 13:14:10.128140  1596 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:10.045878  4599 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.133527  1000 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:10.053824  4473 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.114193  4565 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:10.122592  4498 consensus_meta.cc:287] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.011s
I20260709 13:14:10.140041  1000 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:10.135785   819 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:10.129370  4514 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:10.132081  4407 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
I20260709 13:14:10.149046  1000 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:09.998013  4393 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.166442  1000 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:10.129602  4626 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:10.130198  4615 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.138403  4450 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:10.208981   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 77258.8us
W20260709 13:14:10.201336  4605 log.cc:927] Time spent T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 13:14:10.214337   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:10.215073   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 79375.1us
I20260709 13:14:10.152017  4498 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.207248  4622 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.215907  4628 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:10.226511   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:10.228859  4546 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.231855  4572 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:10.237108   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.232686  4579 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:10.179838  4407 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.252130  4632 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.252696  4632 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.253206  4407 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:10.237433   855 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.254468   996 raft_consensus.cc:2410] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:14:10.232008  4619 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:14:10.219832  4616 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:14:10.257786  4498 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:10.222335  4607 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:14:10.227139  4609 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:14:10.295846  1596 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:14:10.246717   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.247910  4633 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:10.220870  4473 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 13:14:10.258821   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.221053  4626 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.248569  4623 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:10.236740  4142 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:10.302896   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 179351us
W20260709 13:14:10.299118  4611 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:14:10.295409  4616 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.258348  4498 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:10.311630   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:10.314560   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 67925.2us
W20260709 13:14:10.314095   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 348287us
I20260709 13:14:10.305297  4633 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.219007  4599 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.315996   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:10.445817   896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.005s	sys 0.005s
W20260709 13:14:10.447518   896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.005s	sys 0.005s
W20260709 13:14:10.447592   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131658us
I20260709 13:14:10.447784   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:10.448108   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 573005us
I20260709 13:14:10.448607   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.317113   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:10.311757  4482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.001s	sys 0.029s
I20260709 13:14:10.308362  4393 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.364964  4625 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:10.316555  4613 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:14:10.374747  4607 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:14:10.374123  4621 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:14:10.421752  4564 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:10.237788  4628 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:10.519651   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2174) took 1305 ms. Trace:
I20260709 13:14:10.519832  4628 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.520110   850 rpcz_store.cc:276] 0709 13:14:09.214257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.214733 (+   476us) service_pool.cc:224] Handling call
0709 13:14:09.230825 (+ 16092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.792219 (+561394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.519401 (+727182us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54000fe454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:10.519577 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289360512}
I20260709 13:14:10.522957  4591 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:10.452520  4592 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.001s
I20260709 13:14:10.523399  4591 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:10.523694  4592 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.524797   998 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.524314  4591 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.529145  4592 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:10.529274  4632 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.311947  4622 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:10.449385  4453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.003s	sys 0.008s
W20260709 13:14:10.473278   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 156249us
I20260709 13:14:10.530347  4453 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.531335  4356 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:10.531893  4356 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:10.531461  4453 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.532756  4356 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.317296  4603 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.534169  4452 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:10.534041  4603 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.450165  4633 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.433997  4631 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:10.476969   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.484694   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.438037  4627 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:10.470409  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 261017us
I20260709 13:14:10.314000  4142 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.489166  4640 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:10.473889  4482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.018s	sys 0.072s
I20260709 13:14:10.486871  4517 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.495460  4638 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:10.497351  4337 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:10.500919  4612 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
I20260709 13:14:10.503206   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1833) took 1664 ms. Trace:
W20260709 13:14:10.506417  4594 consensus_meta.cc:287] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.004s
I20260709 13:14:10.563386  4420 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:10.563031  4640 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.563937  4420 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:10.563761  4594 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.564612  4640 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.565191  4517 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:10.507297   929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1951) took 1420 ms (client timeout 1854 ms). Trace:
W20260709 13:14:10.566326   929 rpcz_store.cc:269] 0709 13:14:09.086981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.088117 (+  1136us) service_pool.cc:224] Handling call
0709 13:14:10.507066 (+1418949us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400327dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:10.507222 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531536256}
I20260709 13:14:10.516880   922 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.517562  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:10.518086  4502 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:10.514963  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.568168  4502 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:10.483256   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 167143us
I20260709 13:14:10.569119  4502 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.378746  4623 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:14:10.464942  4498 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.563498   849 rpcz_store.cc:276] 0709 13:14:08.838686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.859963 (+ 21277us) service_pool.cc:224] Handling call
0709 13:14:08.878047 (+ 18084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:08.878443 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.878470 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:08.883983 (+  5513us) log.cc:844] Serialized 26 byte log entry
0709 13:14:09.094469 (+210486us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:09.230496 (+136027us) log.cc:844] Serialized 14 byte log entry
0709 13:14:09.533415 (+302919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:09.533511 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.502210 (+968699us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b2400195998. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff8d89c4 00007f0cff8d6acb 00007f0cffa91dc9 00007f0cffa91a55 00007f0cffa90fb0 00007f0d02707488 00007f0d02707be8 00007f0d026df398 00007f0d026da76a 00007f0d026d9bb6
0709 13:14:10.502798 (+   588us) raft_consensus.cc:1685] finished
0709 13:14:10.502942 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.503144 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17180,"prepare.queue_time_us":82374,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4166,"raft.queue_time_us":48926,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":11696,"spinlock_wait_cycles":10276096,"thread_start_us":558163,"threads_started":3,"wal-append.queue_time_us":264342}
I20260709 13:14:10.572180  4337 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.572834  4498 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.487908  3987 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:10.539404   998 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:10.540118  4564 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.570118  4142 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:10.436426  4626 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.406169  4616 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
W20260709 13:14:10.560490   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 75945us
W20260709 13:14:10.561729   997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.046s	sys 0.082s
I20260709 13:14:10.563081  4600 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.584933  4564 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.585208  4600 consensus_queue.cc:1059] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:10.564788  4420 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.565933  4594 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.516759   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 1955) took 1428 ms. Trace:
I20260709 13:14:10.586160  4600 raft_consensus.cc:3055] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:10.534631  4452 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.586486  4420 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.586627  4600 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:10.587198  4452 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:10.587759  4597 consensus_queue.cc:1175] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 12 status { last_received { term: 11 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:10.587670  4600 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.588264  4452 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.589010  4600 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.570261  4502 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.570717   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:10.578004  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 63151.7us
W20260709 13:14:10.592764  4588 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 1.256s	user 0.004s	sys 0.000s
I20260709 13:14:10.593461   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:10.554198  4641 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:14:10.595304  4595 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.963s	user 0.000s	sys 0.001s
I20260709 13:14:10.595350  4594 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.595569  4562 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:14:10.486007  4625 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.597096  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.596191  4594 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.601749  4625 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:10.582998   997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.047s	sys 0.082s
I20260709 13:14:10.603375  4415 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:10.584348   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.607033  4635 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:10.563473  4482 consensus_queue.cc:1243] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:10.582561  4459 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:10.608278  4482 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.559903  4631 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.608603  1595 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:10.608728  4482 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.597536  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.610452  4631 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.610999  4482 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.577834  3987 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.612193  3987 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:10.613291  4514 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.615846  4624 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:10.586381   921 rpcz_store.cc:276] 0709 13:14:09.088474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.206142 (+117668us) service_pool.cc:224] Handling call
0709 13:14:10.516535 (+1310393us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:10.516699 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":885927808}
I20260709 13:14:10.576639  4615 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.619328  4615 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.576857  4565 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.621788  4565 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.622268  3987 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.623289  4603 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:10.661331   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:10.662232   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:10.607591  4635 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:10.562642  4627 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:10.625020  4501 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.629297  4451 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.629426  4246 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.672608  4246 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.674813  4246 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.636540  4617 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.675408  4617 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.677613  4617 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:10.638739   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.679046   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:10.679334   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.685164  4633 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.670835  4631 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.688385  4631 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.689935   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.690421  4631 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.690635   930 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
W20260709 13:14:10.639027   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.692082   923 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:10.640671   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.694321   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.694983   925 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 5.
I20260709 13:14:10.643962  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.696827   995 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:10.697641  4615 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.698164  4615 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.699909  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.700227  4615 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.645586  4639 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.700498  1000 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.700934  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:10.646966  4457 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.701926   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:10.701777  4457 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.702287  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.702719   998 raft_consensus.cc:2393] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:10.703015  1003 raft_consensus.cc:2368] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:10.643199  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:10.646432   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 171080us
W20260709 13:14:10.648979   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 162341us
I20260709 13:14:10.659546   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.703965  4457 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.704835   896 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.705106   921 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:14:10.663616  4552 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:10.639204  4394 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.709784   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.700016  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.700811  4639 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.703750  4393 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.705967   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.706413   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.615427  4514 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:10.736537   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 57578us
I20260709 13:14:10.687438  4633 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.707139   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.707902   850 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:10.708529   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:10.741124   998 raft_consensus.cc:2368] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:10.708583  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.701839  1002 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:10.660406   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:10.742941   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82607.7us
I20260709 13:14:10.744405   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:10.703824  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.715692  4620 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:10.724768  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.752713  4633 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:10.665841  4550 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.753551  4550 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.706941   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.755090   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.665254  4494 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.755856  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.755972  4494 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.758759  4494 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.665105  4482 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:10.759816   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 50141.8us
I20260709 13:14:10.759773  4482 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.760730   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.761610   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.762341  4457 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:10.762720  4457 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.763054  4457 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:10.763271  4634 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:10.748867  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.765098   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.751099   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.750808  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:10.766778   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 448) took 1168 ms. Trace:
I20260709 13:14:10.767143   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.767709  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.768121  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.768589  1002 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:10.769013   925 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:10.769948   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:10.771482   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:10.752925  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.772279   928 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.772328  1000 raft_consensus.cc:2410] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:10.767165   997 rpcz_store.cc:276] 0709 13:14:09.598315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.950521 (+352206us) service_pool.cc:224] Handling call
0709 13:14:09.950897 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:09.951197 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.951235 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:09.965313 (+ 14078us) log.cc:844] Serialized 26 byte log entry
0709 13:14:10.602998 (+637685us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:10.765214 (+162216us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.766123 (+   909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.766554 (+   431us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.766602 (+    48us) raft_consensus.cc:1685] finished
0709 13:14:10.766612 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.766727 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241,"thread_start_us":719159,"threads_started":2,"wal-append.queue_time_us":668765}
I20260709 13:14:10.772740  1004 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.665519  4565 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:10.774403   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.735539  4459 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:10.739367  4502 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:10.742985  4246 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:10.764775  4547 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.751263  4514 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.785537  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:10.706279  4451 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.755486  4550 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.770395  4482 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.792806   821 leader_election.cc:400] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:10.771557  4617 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:10.739051  4393 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.734827  4394 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:10.853528   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60883.4us
I20260709 13:14:10.868544   821 leader_election.cc:400] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6.
I20260709 13:14:10.870069   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:10.797986  1004 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
W20260709 13:14:10.861662   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.861238   928 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:10.765014  4594 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:14:10.874493   928 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:10.788335  4502 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.771039  4616 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:14:10.866636   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:10.790627  4626 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:10.876971   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.794705  4638 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.800141  4644 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:10.902442   821 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:10.887786   928 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:14:10.898729   924 raft_consensus.cc:2368] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:10.903698   998 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
W20260709 13:14:10.905921  4637 log.cc:927] Time spent T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:14:10.865876  4550 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:10.784096  4565 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:10.909968   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 147025us
W20260709 13:14:10.913232   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 175961us
I20260709 13:14:10.921020   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.921092   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.922693   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.923730   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:10.924345   928 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:10.924412   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.787218  4459 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.873495  4617 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:10.874035  4482 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:10.874634  4573 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:10.918644   852 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.851890  4623 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:14:10.919250   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:10.951473   998 raft_consensus.cc:2393] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:10.788715  4599 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:14:10.923020   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.925285   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:10.955500   928 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:10.955560   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.956525   853 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:10.926086   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.955154   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.958734   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:10.960420  1598 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:10.914431  4640 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.878414  4634 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:10.910516  4645 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:14:10.946208   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:10.964323  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:10.908119  4453 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:10.978639   928 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:10.903318  4626 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:10.946894  4415 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:10.992594  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:10.993837  4335 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.994314  4335 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.994946  4336 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.909987  3987 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:10.914753   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 208860us
I20260709 13:14:10.963836  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.964074  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.783034  4635 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.866205  4628 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.926782  4564 raft_consensus.cc:2764] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:14:11.011232   923 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:10.974746   852 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:10.970005   851 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:10.947774   967 leader_election.cc:400] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:11.012311   967 leader_election.cc:403] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:14:10.944774  4616 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:10.979897  4615 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:10.946200  4423 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:10.864701  4627 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:10.979714  4453 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:11.001047  4394 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:10.999420  4335 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:10.947893  4420 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:14:11.002236  4581 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:11.003082  4650 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:11.003304  4356 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.948374  4573 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.004158   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1882) took 1371 ms. Trace:
I20260709 13:14:10.951824  4550 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:11.012410  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.031936   996 rpcz_store.cc:276] 0709 13:14:09.633026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.688914 (+ 55888us) service_pool.cc:224] Handling call
0709 13:14:11.003967 (+1315053us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400359654. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:11.004097 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237993216}
I20260709 13:14:11.014814  4639 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:11.015724   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 68177.6us
I20260709 13:14:11.032635  4550 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.033859   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:11.034713   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:11.035363  4407 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:11.035735  4407 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.035974  4142 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:11.036079  4407 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.036762  4565 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.037529  4600 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:11.038097  4459 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.005748  4545 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:14:11.039229  4600 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.980825   928 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:10.988222  4494 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:11.018621  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:10.907053  4644 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:10.990335  4626 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.992631  4415 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.061924  4644 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.024366  4617 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.004549  4013 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:11.026584  4648 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:14:11.068540  4649 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:11.068847  4013 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:11.026885   923 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:11.069298  4013 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.016103   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.039330  4634 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.039925  4451 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.039979  4319 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.054831   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.019649  4615 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.020102  3987 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
W20260709 13:14:11.060039  4557 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 2.274s	user 0.003s	sys 0.003s
I20260709 13:14:11.061226  4356 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:11.070338   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:11.086862   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.097019  4628 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.072594  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.098434  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.098570  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.023167  4616 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:11.061981  4415 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.054672  4423 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.117517   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.100718  4615 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.128084   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.134266   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.136874  1596 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:14:11.138583  4663 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:14:11.136317  4142 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.058650  4600 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.097924  4603 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:14:11.144491   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.101243  4319 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.144227  4639 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:11.156514  4639 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.158785  4639 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.159315  4657 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:11.159914  4657 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.161384  4457 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.162228   928 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.100693  4453 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.164276  4453 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.169088  4514 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:11.169278  4659 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:11.169427  4013 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.171962  4319 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:11.140341  4628 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:11.141163  4653 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.179641  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 437) took 2628 ms. Trace:
I20260709 13:14:11.142138  4494 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:11.138115  4599 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.180066  1001 rpcz_store.cc:276] 0709 13:14:08.546809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.546944 (+   135us) service_pool.cc:224] Handling call
0709 13:14:08.547262 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:08.547646 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.547682 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:08.548197 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.673062 (+124865us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:08.673117 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.673203 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:11.040758 (+2367555us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:11.155066 (+114308us) raft_consensus.cc:1685] finished
0709 13:14:11.155092 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:11.175260 (+ 20168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":460,"spinlock_wait_cycles":27876736,"thread_start_us":123725,"threads_started":1,"wal-append.queue_time_us":191694}
I20260709 13:14:11.174346  4514 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:11.179122  4013 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.172398  4616 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.196606  4514 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.197894  4514 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.183305  4423 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.202757  4407 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.206988  4597 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.207574  4597 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.208393  4600 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:11.184353  4653 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.188097  4494 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.218935  4626 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.232761  4626 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:11.232784  4661 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.218s	user 0.003s	sys 0.000s
I20260709 13:14:11.266901 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:11.310348   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.173540   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.313459   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.314457   924 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:11.326666  4142 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.333014  4423 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:11.334195  4423 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.326740  4453 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:11.336678  4453 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:11.341389  4672 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:14:11.337884  4617 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.344539  4620 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:11.345021  4673 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:14:11.348047  4581 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.348915  4407 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:11.353915  4494 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.355073   848 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:11.355770   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 1936) took 1531 ms. Trace:
I20260709 13:14:11.356447  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.360075  4617 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.361754  4620 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.228449  4573 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.362898   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.364590   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2179) took 1581 ms. Trace:
I20260709 13:14:11.365717  4453 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.367704  4423 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:11.368533   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.375329   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.377493  4573 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.380927   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.381562   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.374219   855 rpcz_store.cc:276] 0709 13:14:09.782572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.783507 (+   935us) service_pool.cc:224] Handling call
0709 13:14:09.783810 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:10.253926 (+470116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:11.364017 (+1110091us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54000c94d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:11.364348 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042680064}
I20260709 13:14:11.382738   851 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:11.382716   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.377524  4581 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.385632  4393 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.386143   855 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:11.373114   922 rpcz_store.cc:276] 0709 13:14:09.824482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.832966 (+  8484us) service_pool.cc:224] Handling call
0709 13:14:10.516688 (+683722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:10.567839 (+ 51151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:11.355145 (+787306us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b5400319f50. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:11.355421 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571634944}
I20260709 13:14:11.389339   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.389912   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.392665  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.393368  4617 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.393559   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.394359  4386 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:11.394786  4386 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:11.393621  1598 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:11.395294  4386 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.396687   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.396875   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:11.397511   929 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:11.397934  4542 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:11.398381  4542 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.398506  4597 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:11.398798  4542 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.399643   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.400496   924 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.401060   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.402688   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.394932   851 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:11.385430  4620 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.404070   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.408895  4452 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.387951  4581 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.373055   848 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:11.411625   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.411695   848 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.407338  4653 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.412506   855 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:11.405014  4591 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.413614   848 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.413545   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.415019   855 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:11.417598   852 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.415414   850 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.425038  4502 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.434048   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:11.408610   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46104.7us
I20260709 13:14:11.404109   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.441267   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.398506   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.429951   999 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:14:11.444904   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.445035  4457 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.435981  4494 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:11.436113   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.446441  4494 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.447211   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.448993  4494 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.449757   929 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:11.410226   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.389822  4420 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:11.450793   925 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:11.457595   925 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:11.426106   853 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:11.454332  4420 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.455396  4335 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:11.459897  4335 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:11.460738  4420 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.461411  4335 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.462934  4420 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:11.464375  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:11.452322   854 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:11.412065   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.454931   851 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:11.458921   850 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:11.439738   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.452669   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.458333   926 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [8.4-9.5]
I20260709 13:14:11.438261  4542 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.462992  4335 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.492735   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:11.505671   925 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:11.474514   930 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:11.506908   999 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1879) took 1900 ms (client timeout 1754 ms). Trace:
I20260709 13:14:11.454138   922 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:11.475353  4642 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:11.507375   999 rpcz_store.cc:269] 0709 13:14:09.606675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.668879 (+ 62204us) service_pool.cc:224] Handling call
0709 13:14:11.506626 (+1837747us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003e4754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:11.506758 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3734823168}
I20260709 13:14:11.507974  4642 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.508414  4642 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.474618   852 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:11.512732   922 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:11.477607  4626 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:11.513492  4626 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.515200   922 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:11.477156   926 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: [6.5-7.6]
I20260709 13:14:11.480268   819 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:11.481076  1001 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:11.506057  4657 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.524549  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:11.525024   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:11.525911  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.526574   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.527518   995 raft_consensus.cc:2393] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:11.528314   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.528561   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.532526   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.534474   847 raft_consensus.cc:2368] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6.
I20260709 13:14:11.536042   928 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:11.539915  4572 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:11.540871  4572 consensus_queue.cc:1243] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:11.541455  4572 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:11.541673   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.541919  4572 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.543069  4572 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.543699  4573 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.478724   855 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:11.546010   923 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 13:14:11.539394   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.565164  4386 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:11.569100   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.570098  4642 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.571228  4581 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:11.573988   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.574520  4597 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:11.570412   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:11.574677   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.544721  4572 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.585757   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.590368   921 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:11.592929   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:11.593431   848 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:11.544405  4573 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.600325   921 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.601430  4386 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.546480  4626 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.606171   930 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:11.575147  4597 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.609512   927 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:11.593793   848 pending_rounds.cc:85] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:14:11.611944   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:11.602540   922 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:11.616935   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.595409   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:11.617722   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.613658   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.624639   853 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.619379   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.620101   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.616781   848 pending_rounds.cc:107] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:11.628656   848 raft_consensus.cc:2889] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:11.627671  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:11.629315   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.615942  4657 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:11.459441  4615 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.637989   922 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:11.628445   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:11.641393  4459 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:11.643718   996 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:11.644577   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.645644  4638 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:11.646440  4638 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.649458  4638 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:11.649998   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 186442us
I20260709 13:14:11.650465  4642 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:11.643290   852 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
W20260709 13:14:11.652431   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.655925   921 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:11.659880  4640 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:11.660354  4640 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:11.660754  4640 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:11.660949  4616 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:11.661526  4616 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:11.662531  4616 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:11.663794   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 190835us
I20260709 13:14:11.664741   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.666368   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.667943  4502 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:11.668213   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44072.3us
I20260709 13:14:11.669250   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:11.673961   847 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.673204  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.673586  1000 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.679399   921 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.677537   995 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:11.678874  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:11.682948  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.683424   854 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:11.693323  4638 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:11.694831   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.695626   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:11.677740  4626 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.699039   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:11.700054  1003 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:11.700770  4494 raft_consensus.cc:2764] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:14:11.700663  4653 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:11.701606  4494 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:11.703094   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 125109us
I20260709 13:14:11.708048   848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1836) took 1868 ms. Trace:
I20260709 13:14:11.713191   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:11.708511   848 rpcz_store.cc:276] 0709 13:14:09.839506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.839724 (+   218us) service_pool.cc:224] Handling call
0709 13:14:09.839973 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.354714 (+1514741us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400290a20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b411c0 00007f0d02b42231 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c
0709 13:14:11.686809 (+332095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:11.707990 (+ 21181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773998976}
I20260709 13:14:11.723986   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.735018   926 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.707242  4142 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:11.703349  4542 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.737864   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:11.756457   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 137963us
I20260709 13:14:11.757254   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.765252  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:11.765609   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:11.766484  4507 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:14:11.766686   995 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:11.767149  4626 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:11.768209   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.769402  4657 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:11.769904  4657 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.769804   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:11.770735   995 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.749495  4142 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.771952  4142 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.774250   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.741360  4620 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:11.765965  4451 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:11.717881  4615 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.780123  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:11.775561   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:11.765686   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63527.8us
W20260709 13:14:11.790343   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 131679us
I20260709 13:14:11.781531  1002 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:11.798240  1003 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:11.799854   996 raft_consensus.cc:2410] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:14:11.815117   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 65334.8us
I20260709 13:14:11.831687   854 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:11.838845   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:11.786278  4573 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.816816  4620 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:11.848958   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 125359us
W20260709 13:14:11.849649   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.847884  4386 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.869846   995 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:11.832832  4591 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:14:11.871899   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:11.874133   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 179411us
W20260709 13:14:11.888118   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60274.4us
I20260709 13:14:11.891059   924 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:11.870395  4386 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.891716   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:11.875686  4415 consensus_meta.cc:287] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.006s
I20260709 13:14:11.894166  4640 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.898178   929 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:11.853930  4407 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:14:11.905761  4552 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.909904   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:11.910665   895 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:11.914812   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.909536   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:11.922392   846 raft_consensus.cc:2410] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:11.924770   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:11.930586   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.931845  4638 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:11.933624   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79810.5us
I20260709 13:14:11.916826  4597 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:11.932435  4638 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:11.935148  4616 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:11.935243   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.935994   848 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:11.930455  4615 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:11.931422  4552 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:11.916174  4415 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:11.951498   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58697.9us
I20260709 13:14:11.936477  4542 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:11.956297  4615 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.958361   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.960083   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.960485   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:11.960805   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.944734  4547 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.968221   852 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:11.942332  4616 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:11.974290  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:11.975278   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.974383   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.980595   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.986874   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:11.987037  4453 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:11.987754  4620 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:11.987995  4453 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:11.988595  4451 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:11.993387  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:11.994019  1598 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.994693   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:11.995172   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:11.993752   850 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.996831  1595 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.964748  4542 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:11.966605  4552 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:11.998138   855 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:11.998421   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 88609.2us
I20260709 13:14:11.996985  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:11.976972  4616 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:12.002560   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66081.4us
W20260709 13:14:12.003547   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.006196   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.007761  1596 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:11.963032  4142 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.016351   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.016991   848 raft_consensus.cc:1813] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Handling vote request from an unknown peer 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:12.017410   848 raft_consensus.cc:2368] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:12.025522   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.026859   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:12.010258  4420 consensus_meta.cc:287] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.004s
I20260709 13:14:12.026995   896 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.030844  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.036383   853 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:11.993924  4615 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:12.040208   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.043839   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:12.045571   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.045440   895 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.046862  1001 raft_consensus.cc:2410] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:12.048234   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:12.049173   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 176236us
I20260709 13:14:12.049618   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:12.050671   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 69501.3us
I20260709 13:14:12.052774   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.056591   851 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:12.057262  4638 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.057566  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:12.058416  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 62576.7us
I20260709 13:14:12.058730  4307 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:12.059289  4307 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:12.060364  4307 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.030356  4453 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.034986  4597 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:12.072628  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:12.073897   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.093288   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.076192   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:12.071576  4680 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:14:12.046973  4494 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:12.123158  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 50670.2us
W20260709 13:14:12.135838   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 90491.6us
W20260709 13:14:12.127759   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 97880.2us
I20260709 13:14:12.093888  4386 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:12.138085  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.138396  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.104668  4420 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:12.139966   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 63987.1us
I20260709 13:14:12.139915  4420 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.141103   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:12.141335  4683 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
I20260709 13:14:12.142608  4420 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.143095  4545 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:12.143898   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 95788us
I20260709 13:14:12.144701   896 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:12.145403  4594 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:12.146299  4507 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.148082  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:12.143529  4545 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:12.152196  4545 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.152835  4620 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:12.153877  1595 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:12.152272   853 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:12.157464   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.157697  4450 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:12.160152   853 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:12.167905  4552 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.170115   855 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:12.176163  4502 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:12.176795  4502 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:12.177647  4502 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.180817  4615 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.077499  4542 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.182581  4542 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.181983   753 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 3 to 7, term changed from 8 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to <none>, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted, no longer has a pending config: VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1), VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3), VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:12.185961   847 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:12.075485  4580 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:12.188050  4580 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.190203  4580 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:12.190385   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 43648.6us
W20260709 13:14:12.193372  4572 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.004s
I20260709 13:14:12.087713  4653 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:12.068395  4453 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:14:12.199932  4453 consensus_queue.cc:1243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:14:12.200698  4573 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:12.201114  4573 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.201434  4626 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:12.201879  4626 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:12.201974  4573 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.142676   818 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:12.076463  4640 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.076761  4597 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.204346  4597 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.204425  4626 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.206122  4626 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:12.208875  4415 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:12.131827  4638 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.211242  4638 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.212001   895 leader_election.cc:400] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
W20260709 13:14:12.213651   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71156.5us
I20260709 13:14:12.214772   848 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:12.151860  4657 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.215575  4514 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:12.230280  1000 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 13:14:12.230636  1000 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:12.196635  4685 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.217456   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.067571  4547 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:12.078833  4142 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:12.227198   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83712.8us
I20260709 13:14:12.227737   850 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
W20260709 13:14:12.236675   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 92050.6us
I20260709 13:14:12.237674   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.234930   753 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:12.240929   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.134662  4494 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.262917   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:12.265213   927 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:12.256702   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:12.244496  4685 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.247215  4494 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.272001  4685 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.272877  4620 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:12.275787  4514 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:12.279062  4494 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.279794   753 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:12.244031  4393 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:12.242053  4451 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:12.240208  4653 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:12.284816  4393 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.292218  4620 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:14:12.306320  4393 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.307063  4620 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:12.307433  4620 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:12.303807  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:12.300195   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58307.8us
I20260709 13:14:12.314756  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:12.315336  1598 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
W20260709 13:14:12.317399   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 76543.2us
I20260709 13:14:12.308048  4653 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.296132  4638 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.288491  4457 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.322242  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.305325  4451 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:14:12.323734  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.356389   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:12.332229  4459 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:12.317595   753 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 10 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:12.331686  4386 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:12.322129  4573 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:12.293354  4615 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:14:12.331364  4545 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.330796  4616 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:12.361725   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.340863  4494 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.315863  4620 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.372058   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.373032  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:12.373835  4688 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.373667  4501 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:14:12.376260  4657 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:12.377270   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51885.2us
I20260709 13:14:12.379228  4638 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:12.447809  4638 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.448340  4638 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.360718  4459 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:12.387816   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.387952  4450 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:12.452548  4620 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:12.387816  4626 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:12.347802  4594 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:14:12.362509  4457 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.324961  4542 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:14:12.387348   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:12.353355  4685 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.438728  4688 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.439847  4494 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.445927  4545 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.451853  4459 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.455291   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.473321  4685 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.473825  4457 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.474383  4494 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.475867  1000 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:12.480603  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:12.488320   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.491492   998 raft_consensus.cc:2410] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:12.493017   753 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 4 to 5, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:12.483822  4450 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:12.490741  4457 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:12.515522   998 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:12.481673  4657 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:14:12.522028   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 134812us
W20260709 13:14:12.522636   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 232483us
I20260709 13:14:12.525792   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.527390  1004 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:12.527164  4657 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:14:12.535207   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.521009  4452 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:12.499822  4597 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.544471  4452 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:12.546326  4452 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.547166  4597 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.547645   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.548286   853 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.513095  4547 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:12.539873   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.550621  4547 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:12.551638   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.551563   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.552457  4547 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.553843  4547 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:12.540710   753 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 6 to 7, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 7 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:12.556561  4494 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.557179   853 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:12.534678  4393 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.560364   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.561081   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.561684  4393 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.563742  4692 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:14:12.564849  4692 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:12.565200  4692 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.565367  4457 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:12.566097  4692 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.566689   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.567553   854 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:12.572090   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:12.579016   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.581287   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.582365  4457 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:12.583278  4457 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:12.521615  4450 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:12.584340  4457 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.584406  4450 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.586535  4653 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.589476  4685 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.590423  4616 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:12.601709  1000 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:12.567349  4692 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.527855  4615 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:12.535979  4335 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:12.543905  4142 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:12.618527  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:12.625047  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 231764us
I20260709 13:14:12.627144   854 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:12.611209  4615 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:12.612895  4335 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:14:12.644354   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 272393us
I20260709 13:14:12.640895  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.542953  4386 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:14:12.618991  4142 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.656131  4142 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.656509  4335 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.657454  4142 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:12.666672  4688 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:12.669503   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.670073   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.670830   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:12.671062   997 raft_consensus.cc:2393] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:12.671952  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.672163  4547 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:12.672607  1001 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:12.672708  4547 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.673254  4547 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.673715  4685 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:12.674291  4685 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.675208  4685 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.649987   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.680727   846 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.640312  4617 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.685750  4459 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:12.693210   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.693397   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 70185.5us
I20260709 13:14:12.694355   969 leader_election.cc:400] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6.
I20260709 13:14:12.694677   969 leader_election.cc:403] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.694556  4335 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:12.616470  4552 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:12.654206  4615 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.625995  4597 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:14:12.702028   846 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:12.713722  4594 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.724189  4594 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.724895  4597 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:12.725479  4597 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:12.725803  4597 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.726624  4594 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:12.726714  4597 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.728338   926 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:12.663393  4638 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.739305   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.713881  4688 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:12.745572  4699 raft_consensus.cc:2764] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:14:12.738600  4617 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:14:12.751369   852 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:12.749050   896 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:12.751145 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:12.693409  4547 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.774344  4547 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:12.765563  4597 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.732141  4552 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:12.776386  4706 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:12.776640  4552 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:12.767995  4307 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:12.774878  1004 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:12.770923   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:12.780553   848 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:12.761758   818 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 2.149s (SENT) [suppressed 18 similar messages]
W20260709 13:14:12.791077   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.793768  4457 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.794328  4457 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:12.796598  4457 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:12.799530  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 251255us
I20260709 13:14:12.765136  4459 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:12.809372  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 168753us
I20260709 13:14:12.810225   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:12.844472  1567 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 2.122s (SENT)
I20260709 13:14:12.846962  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:12.811344  4547 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:12.792003  4620 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:14:12.829107   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:12.854293   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 44146.3us
W20260709 13:14:12.854352   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 54458.4us
I20260709 13:14:12.856781   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:12.821312  4452 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:12.824100  4615 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.876844  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.879735   848 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:12.883219   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:12.882690  1004 raft_consensus.cc:2368] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9.
I20260709 13:14:12.883841   846 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:12.884244   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 122503us
W20260709 13:14:12.894558  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50136.1us
I20260709 13:14:12.895928  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.896586  1001 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:12.903635  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.903743   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:12.904505  1001 raft_consensus.cc:2410] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:12.904851   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:12.905306  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:12.905897  1001 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.905966   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.904526   995 raft_consensus.cc:2393] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:12.879593   753 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 6 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 9 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:12.872824  4573 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:12.859171  4704 consensus_queue.cc:1059] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:14:12.872489  4542 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:12.906626   996 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:12.908579  4547 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:12.922561  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.944631  1002 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:12.918165  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:12.923251   999 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:12.924764  4514 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:12.925004  4631 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:12.945665  1003 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:12.947827   846 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:12.953756   995 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:14:12.913739  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 161491us
I20260709 13:14:12.961889  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:12.962932   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 133943us
I20260709 13:14:12.916834  4573 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:12.967648  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.975199  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:12.976013  1596 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:12.984880   927 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:12.985536   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:12.966221  4712 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
W20260709 13:14:12.974609  4711 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:14:12.956542  4640 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:12.957166  4542 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:12.996011   922 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:12.988134   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:12.997363   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:12.999817  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:12.972946  4573 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.005705  1601 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:13.004403   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.011102  1598 raft_consensus.cc:2393] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:12.956573  4514 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.020691   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.025370   924 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.028247  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:13.028735  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.003592  4704 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:13.028891  1599 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:13.034760  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.037757  1002 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.038801   921 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:13.039772  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.041723   996 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:13.042135   996 pending_rounds.cc:85] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 6
I20260709 13:14:13.048143  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.050558   996 pending_rounds.cc:107] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:13.050976   996 raft_consensus.cc:2889] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:13.050001  1004 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:13.055886   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 223) took 7905 ms. Trace:
I20260709 13:14:13.043174  4626 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.056741  4626 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.056294   923 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.062125   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.065644  4626 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.067013   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.067692   929 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:13.068581   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.069314   923 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:13.073614  4721 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:14:13.073788  1003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:13.074666  1003 raft_consensus.cc:606] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.075037  1003 raft_consensus.cc:614] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:13.077775   924 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:13.071394   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.017580  4640 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.086618  4726 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:13.093930  4542 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.097155  4514 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:13.097368  4620 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:13.082931   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.084117   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:13.107240  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:13.023134  4719 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:14:13.117712  4688 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 5. Current term is 6.
I20260709 13:14:13.053570  4704 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:13.118656  4450 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.124037   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52773.6us
I20260709 13:14:13.130837  1001 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:13.134676   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 50658.5us
I20260709 13:14:13.141402   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.135787   997 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.145947   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.147012   996 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.159971  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.161525  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.162256  1001 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:13.165526  4514 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.166343  4514 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.169378   896 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.175179   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.135898  4700 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.187422   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.140396  4393 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:13.056289   749 rpcz_store.cc:276] 0709 13:14:05.048128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.051231 (+  3103us) service_pool.cc:224] Handling call
0709 13:14:05.308841 (+257610us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b5400020710. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1b0b2 00007f0d08994a89 00007f0d0892cda6 00007f0d088fe088 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc
0709 13:14:12.922859 (+7614018us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:08.871133 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:08.962631 (+ 91498us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:08.963489 (+   858us) write_op.cc:435] Acquired schema lock
0709 13:14:08.968110 (+  4621us) tablet.cc:662] Decoding operations
0709 13:14:08.970520 (+  2410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:08.970604 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:08.970646 (+    42us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:14:08.998316 (+ 27670us) tablet.cc:701] Row locks acquired
0709 13:14:08.998348 (+    32us) write_op.cc:260] PREPARE: finished
0709 13:14:08.998565 (+   217us) write_op.cc:270] Start()
0709 13:14:08.998734 (+   169us) write_op.cc:276] Timestamp: P: 1783602848998499 usec, L: 0
0709 13:14:08.998767 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:14:09.059252 (+ 60485us) log.cc:844] Serialized 9597 byte log entry
0709 13:14:10.255653 (+1196401us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3ae28 00007f0d06575581 00007f0d06574d58 00007f0d065799f1 00007f0d065799a9 00007f0d06579939 00007f0d06579901 00007f0d06578b2d 000000000038f354 000000000038f199
0709 13:14:10.589775 (+334122us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:10.688008 (+ 98233us) op_driver.cc:464] REPLICATION: finished
0709 13:14:10.904950 (+216942us) write_op.cc:301] APPLY: starting
0709 13:14:10.907179 (+  2229us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:11.054838 (+147659us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:11.057367 (+  2529us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:11.233201 (+175834us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:11.233769 (+   568us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:14:11.233847 (+    78us) write_op.cc:312] APPLY: finished
0709 13:14:11.234539 (+   692us) log.cc:844] Serialized 164 byte log entry
0709 13:14:11.234793 (+   254us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:11.235107 (+   314us) write_op.cc:454] Released schema lock
0709 13:14:11.235204 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:11.237730 (+  2526us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:11.237782 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14260,"reported_tablets":50,"spinlock_wait_cycles":82264832,"child_traces":[["op",{"apply.queue_time_us":184180,"apply.run_cpu_time_us":11585,"apply.run_wall_time_us":356578,"num_ops":24,"prepare.queue_time_us":65090,"prepare.run_cpu_time_us":84942,"prepare.run_wall_time_us":1397404,"raft.queue_time_us":77111,"raft.run_cpu_time_us":14787,"raft.run_wall_time_us":164505,"replication_time_us":1689086,"spinlock_wait_cycles":63844992,"thread_start_us":290206,"threads_started":4,"wal-append.queue_time_us":181066,"wal-append.run_cpu_time_us":8603,"wal-append.run_wall_time_us":3105152}]]}
I20260709 13:14:13.188534   996 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.190726   998 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.176605   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.143235  4631 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:14:13.196130   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.202688   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.179977  4653 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:13.133548  4726 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.203508   996 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.176352  4451 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.177390  4450 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.209812   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.192481  4393 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.210701   997 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:13.209245   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 245553us
I20260709 13:14:13.213140  4726 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.199010  4494 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:13.215690   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.218066  1000 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:13.219772  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.240514  1000 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.219882  1001 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:13.208489  4688 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.249059   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.250200   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.252171  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.216497  4393 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:13.242233  4700 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:13.254652   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.237681  4494 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.214172  4451 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.258669   929 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:13.260373   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.260833   925 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:13.261850  4594 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:13.270383  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.271003   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.261860   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.273335   851 raft_consensus.cc:2410] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:13.277350   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.280767   997 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:13.257118  4700 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.280251   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.284121   855 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:13.282316   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.286175   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.286916   848 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.259091  4494 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:13.288174   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.282385   850 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:14:13.289426   850 pending_rounds.cc:85] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:14:13.289765   850 pending_rounds.cc:107] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:14:13.290046   850 raft_consensus.cc:2889] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:13.282651   846 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:13.282850   849 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:14:13.291575   849 pending_rounds.cc:85] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 7
I20260709 13:14:13.291922   849 pending_rounds.cc:107] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:14:13.292205   849 raft_consensus.cc:2889] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:13.281212  4724 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.283411  4653 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:14:13.293995  4653 consensus_queue.cc:1243] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:13.293922  4724 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.300201  4625 raft_consensus.cc:3055] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:13.300712  4625 raft_consensus.cc:740] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:13.301716  4625 consensus_queue.cc:260] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.302229   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.302858  4625 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.303370   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:13.303915  4726 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.304385  4726 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.304991   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.306357  4692 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:13.306977  4692 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:13.307510  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.308521  4720 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.309116   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.309036  4720 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.310523   928 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:13.300209  1001 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.313088   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.314345   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.294027   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.308524  1598 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:13.315487  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.318027   928 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:14:13.314369   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65396.9us
I20260709 13:14:13.321067   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.321830  1000 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:13.257180  4732 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:13.315380   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.318037  1001 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:13.325160   927 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:13.252933  4688 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.326624   999 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:13.272114  4514 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.328411   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.330183   854 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:14:13.342355   854 pending_rounds.cc:85] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 4
I20260709 13:14:13.342625   854 pending_rounds.cc:107] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:14:13.342849   854 raft_consensus.cc:2889] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:13.324990   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.343257   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.332180   925 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:13.338095  4699 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.293331  4734 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:14:13.345353  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:13.347096   855 raft_consensus.cc:2393] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:13.347856  1001 raft_consensus.cc:606] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.348215  1001 raft_consensus.cc:614] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:13.287586  4700 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:13.311419  4720 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.280516  4657 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.360375   848 raft_consensus.cc:2410] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:13.350805   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.359378   999 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:13.325662  4732 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.342937  4547 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:13.367401   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.249859  4704 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.368395   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.369774   848 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:13.371361   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.352098  4514 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.363507  4640 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:13.328521  4694 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:13.363835  4386 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.352231  4594 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:14:13.369027  4732 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:13.386765  4735 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:14:13.367874  4142 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.363759  4626 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:14:13.387461  4692 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:13.392387  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.367305  4657 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.391782  4459 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:14:13.280259  4631 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.419556   855 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 13:14:13.422878   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56652us
I20260709 13:14:13.379953  4638 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.355295  4699 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.375437  4552 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.386759  4451 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.428205   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:13.385670  4640 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:14:13.415894  4692 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:13.432832  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:13.436846  1598 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:13.413003  4386 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.409500  4142 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:13.433526  4740 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:14:13.434163  4699 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.393543  4452 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:14:13.452260  4736 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.389879  4393 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.449954   854 raft_consensus.cc:1240] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:13.495885   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 67981.6us
I20260709 13:14:13.497071   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:13.500083   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:13.504911  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 76155.5us
I20260709 13:14:13.505918  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.507685   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.508558   849 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:13.511291  4393 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:13.511392  4386 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.511912  4142 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.512374  4452 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.515508   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.516745   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.519753  4725 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:13.519928  4741 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.084s	user 0.000s	sys 0.002s
W20260709 13:14:13.522888   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.525512   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.526564  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.527087   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.527180  1001 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:13.430485  4734 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:14:13.430184  4737 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:13.437223  4450 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.536348   849 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:13.539450  4657 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.543701  4386 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:13.543957  4692 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.494782  4736 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.549505   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.550755  4737 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.554339  4724 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.431165  4552 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.567180  4494 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:13.534900  4615 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.001s
I20260709 13:14:13.504375  4638 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:13.535830  4594 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:14:13.588876  4688 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.535455  4142 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:14:13.594918  4715 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:13.683419  4729 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.431s	user 0.003s	sys 0.004s
I20260709 13:14:13.608371   849 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:14:13.610385  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 104579us
I20260709 13:14:13.686081  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.689018  4709 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.683467  4715 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:13.690980  4715 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.692513  4494 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.694154   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.645334  4719 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.610663  1000 pending_rounds.cc:85] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 3
W20260709 13:14:13.611361  4573 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.003s
I20260709 13:14:13.609939  4700 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.612414  4726 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.614738  1001 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:13.615347   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.615571   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.547394  4742 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:14:13.547410  4694 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:13.619380   995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:13.707638   848 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:13.708168   995 raft_consensus.cc:606] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.623589  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:13.624202  4450 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.624977   819 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:13.713356   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 88546.2us
I20260709 13:14:13.643846   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.649158  4750 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.656155  4737 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:13.661585   851 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:13.668294   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:13.670639   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 136906us
I20260709 13:14:13.717022   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.718592   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.659806   967 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:13.677616   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.720714   854 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:13.682005   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.695794  1000 pending_rounds.cc:107] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:14:13.721455  1000 raft_consensus.cc:2889] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:13.721647   850 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:13.670197  4638 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:13.703914  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.662360  4692 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:13.710400  1595 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:13.676620  4453 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.711294  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 87785.7us
W20260709 13:14:13.701710   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 300529us
I20260709 13:14:13.724115   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.679992  4615 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.731272   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:13.738554   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.674497  4542 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:14:13.676139  4626 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:14:13.739156   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.762462   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.727885  4638 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:13.730126  4594 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:13.736063  4726 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:13.706398  4573 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.727883  4450 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.743064  4597 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:13.785535  4724 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.763201  4142 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:13.797948   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 129767us
I20260709 13:14:13.740446  4615 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.790329  4494 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:13.737409  4453 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:13.809806  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80267.9us
I20260709 13:14:13.784847  4594 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.706842  4719 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.809819  4620 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:14:13.816322  4625 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.006s
I20260709 13:14:13.819262  4494 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:13.820313  4494 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.803290  4742 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:13.731676  4750 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.807545  4726 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:13.797226  4451 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 13:14:13.852489   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53759.6us
I20260709 13:14:13.825109  4594 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:13.809667  4142 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:13.860699  4688 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:13.811442  4694 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:14:13.882632  4573 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.883956  4552 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.884737  4719 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:13.876048   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 80336.8us
I20260709 13:14:13.877555  4709 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.878484  4450 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.879307  4750 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:13.872505  4743 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:13.857227  4725 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:13.896358   995 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:13.905925   922 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:14:13.906267  4453 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.906628   922 raft_consensus.cc:606] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.907068   922 raft_consensus.cc:614] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:13.898069  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.875944  4688 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:13.848825  4732 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:13.907296  4751 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:14:13.898557  4743 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:13.911769  4743 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:13.894740  4709 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.916683  4688 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:13.918094  4688 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:13.919399  4700 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.920640  4615 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.892374  4742 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:14:13.921247  4732 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:13.922372   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.922214  4142 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:13.923031   853 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:13.924547  1566 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:13.925961  4756 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:13.930547   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.890398   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.938105   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.941318  1001 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:13.941818   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.942659   851 raft_consensus.cc:2410] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:13.942806   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:13.943122   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.943950   854 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:13.943987   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53676.1us
I20260709 13:14:13.944859   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.946043   853 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:13.946520  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.953123  1001 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:13.947901   848 raft_consensus.cc:2410] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:13.949085   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:13.953894  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 634) took 1021 ms. Trace:
I20260709 13:14:13.949687   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.951116  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:13.952020   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 189397us
I20260709 13:14:13.954329  1595 rpcz_store.cc:276] 0709 13:14:12.932610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.956786 (+ 24176us) service_pool.cc:224] Handling call
0709 13:14:12.976605 (+ 19819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.725879 (+749274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.953687 (+227808us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b5400270954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:13.953842 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687012096}
I20260709 13:14:13.939838  4640 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:13.954702   995 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:13.958150   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.959335  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.958760   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.964347  1003 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:13.964865   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:13.965525  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.966367  1004 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:13.956580   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 296850us
I20260709 13:14:13.945343  4725 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:13.967424   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:13.974215   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:13.959777  4742 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.961908  4453 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:13.983624   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.993330   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.994392   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.994659   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:13.995352   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.000013   921 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:14.000563   928 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:13.947187  4700 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:13.999550   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.002327   997 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:14.001255   925 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:14.000613   924 raft_consensus.cc:2376] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11. Re-sending same reply.
I20260709 13:14:13.950414  4615 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:14:14.001463   929 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:14.001243   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.005502   921 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:13.973997  4725 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.008653  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:14.009651  1002 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.009999  1002 raft_consensus.cc:614] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:14.013451   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.014390  1001 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:14.015002  4767 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:13.979015  4750 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:14.018878  4767 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:14.015723  1595 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:14:13.976897  4626 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:14.016012  4768 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:14:14.016336  4758 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:14:14.017046  4453 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:14.039364   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57848.1us
I20260709 13:14:14.017439   927 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:14.040482   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:14.015633   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.020474   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.042673   923 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:14.042958   996 raft_consensus.cc:2410] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:14.017470  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:13.996484  4547 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.037285   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:13.979288  4450 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:14:14.046892  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.046013   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.021443  4725 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.049500  1001 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:14:14.049415   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73658.4us
I20260709 13:14:14.045513  4547 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.015013  4774 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:14.055883   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.056720   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.057267   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.057302   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.058750   925 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:14.059718   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:14.027159  4451 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:14:14.057834   927 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:14.002332  4742 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.072033  4784 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:14.072242  4742 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.069334   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.073175  4739 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.073850  4771 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:14.074025  4737 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:14.074597  4771 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.075045  4771 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.069551  4638 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.083549  4638 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.072711  4784 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.084172   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.073802  4739 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.090297   996 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:14.090806  4626 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:14.090848  4774 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:14.055915  4547 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.034776  4685 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:14:14.092147   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.094060  4743 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.095515   849 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:14.100015   967 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.104012  4783 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:14.104194  4573 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:14.104624  4573 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.105002  4573 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.107683  4719 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:14.108167  4719 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:14.108619  4719 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.086192  4784 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.068060  4750 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:14.111821  4750 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.112197  4750 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.117838   997 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:14.121891   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.098999  4626 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:14.096630  4734 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:14.129936   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 74299.3us
I20260709 13:14:14.133106  4758 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:14.134074  4594 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.135962  4386 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:14.136446  4386 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.136838  4386 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.144312   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:14.150120   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:14.153918   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 140650us
I20260709 13:14:14.158030   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.166923   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.153769  4758 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:14.150278  4781 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:14.169885   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.156461  4783 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:14.175117   850 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:14.167907  4737 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:14.177593  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.170450  4709 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:14.206420   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 166048us
I20260709 13:14:14.172353  4758 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.219841  4758 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.207803  4786 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.161463  4547 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.208282  4787 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:14.224583  4787 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.226234  4547 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.226838  4787 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.211736  4778 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:14:14.227442  1001 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:14.212268  4743 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.229113   997 raft_consensus.cc:606] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.229686   997 raft_consensus.cc:614] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:14.230773   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.148043  4734 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.233052   927 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:14.214574  4719 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.235359   850 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:14.217307  4594 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.223901  4786 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:14.184973  4781 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.214915  4742 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.218200  4771 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.239936   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.240402  4778 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:14.243111  4719 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.244202   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.255996   928 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.258304   846 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:14.258780   846 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:14.250756   821 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:14.206239  4784 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:14.205832  4739 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.259752   846 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.238835  4737 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:14.271552  4737 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.269392  4594 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.273567  4594 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.273767  4737 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.273957  4594 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.260780  4778 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:14.274143  4386 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.274923  4778 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.264669  4784 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:14.276168  4784 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.276258  4778 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.249554  4786 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.280040  4725 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.282349  4725 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.265230   821 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:14.284538   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.268909   846 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.249312  4783 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:14:14.291891   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.293177   930 raft_consensus.cc:2393] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:14.297457   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.304196   850 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.172407  4626 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.308060  4626 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.308161  4573 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.311283  4573 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.311825   850 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:14.314186  4386 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.314570  4788 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:14.315184  4788 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:14.317970  4779 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 13:14:14.318540   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:14.319118  4746 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:14.319950  4780 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:14.320657  4780 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:14.321147  4780 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.322530  4771 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.320045  4746 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:14.266922  4734 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.267088  4742 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.326814   928 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:14.334353   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.323912  4746 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:14.350889  4746 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.352092  4746 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.353101   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.337275   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.353719   930 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.354075   927 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.352404  4786 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.341495  4750 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.341764  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:14.343832  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:14.362916  4750 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.363627  4750 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:14.364102  4750 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:14.365029  4750 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.346174  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:14.366914 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:14.367342  1599 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.346925  4788 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.349660   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.370769   923 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.373746  4736 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.377161  4719 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.377486  4694 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.377959  4719 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.383671  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:14.385995  4742 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.378105  4694 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.386011  4781 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:14.389488   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 411335us
I20260709 13:14:14.390762  4694 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.391054   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.398821   969 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.399665  4386 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:14.400401  4386 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:14.402127   969 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.402016  4386 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:14.402983  4726 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:14.403635   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 75432.9us
I20260709 13:14:14.399644   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.404716   821 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:14.404799   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:14.405771  4453 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.406195  4453 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.406538  4453 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.406801   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.407716  4386 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:14.408105  4386 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:14.394937   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 225132us
I20260709 13:14:14.408465  4386 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:14.408990   896 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.410228   896 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.411043  4734 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:14.381945  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:14.413540   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.414877   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.415723   848 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:14.416298  1596 raft_consensus.cc:2368] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:14.417364   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.418747   896 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.420008   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.421155   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:14.424899   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.427532   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.428385   924 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.429368  4796 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.429884  4796 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.432375  4688 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:14.412447  4742 consensus_queue.cc:1059] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:14.433530   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.440896   849 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:14.440912   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.439560   924 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:14.425012  4615 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:14.424721  4734 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:14.443799   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.444077   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.444375  4734 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:14.444674   847 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:14.444732   852 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:14.445307  4734 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.444381   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.446460  4796 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.447449   967 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.449048  4694 raft_consensus.cc:2804] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:14.449640  4694 raft_consensus.cc:697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:14.450556  4694 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.451066  4719 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.451267   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.452054   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.452482   854 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:14:14.452672   922 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:14.452922   854 pending_rounds.cc:85] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
I20260709 13:14:14.453313   854 pending_rounds.cc:107] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:14:14.453605   854 raft_consensus.cc:2889] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.452078   997 raft_consensus.cc:2368] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:14.454391  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.455336   855 raft_consensus.cc:2368] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:14.455483  1001 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:14.454624   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.460191   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.463243   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.464320   854 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:14.425604   927 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:14.426219  4700 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:14.476701   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.478327   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:14.479166   921 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.481431   894 leader_election.cc:400] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:14.481899   894 leader_election.cc:403] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:14.482688   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.484575   846 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:14:14.484743   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.485716   854 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.485177   846 pending_rounds.cc:85] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 8
I20260709 13:14:14.487354   846 pending_rounds.cc:107] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:14:14.489984   846 raft_consensus.cc:2889] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.455772  4732 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:14.500568   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.496752   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.472438  4685 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.501518   846 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.478158  4781 raft_consensus.cc:2764] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:14:14.486856   930 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:14.457433  4688 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:14.500417  4514 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:14:14.494936  4142 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:14.457688  4758 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:14.521212   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.522145   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.505081  4685 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.525701   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.527385   846 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:14.535395   930 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:14.536965   849 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:14.530375  4685 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.539809   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.564170   930 raft_consensus.cc:1240] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:14.564692   922 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:14.541185  4786 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.531837  4780 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.544417  4758 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.548910   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.524142  4653 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.552644   921 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:14.589777  1564 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:14.520007  4736 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:14.554925   923 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:14.539774   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:14.929476   846 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:14.552242   854 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
W20260709 13:14:14.608736  4798 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:14:14.729524  4803 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:14:14.798938  4799 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.121s	user 0.000s	sys 0.003s
I20260709 13:14:14.818816   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:14.836319   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:15.265161   847 raft_consensus.cc:3060] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:14.917431 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:14.959762   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:14.616410   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94398.5us
I20260709 13:14:14.755311   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:15.272900  4763 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:14:15.273962   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 518687us
I20260709 13:14:15.274204  4335 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:15.274412  4763 consensus_queue.cc:1059] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:15.275080   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:14.565759   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:15.276193  4335 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:15.276592  4335 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:14:15.276690   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 710970us
I20260709 13:14:15.277511  4335 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.278002   895 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:15.278754  4335 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:15.279047   895 leader_election.cc:400] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:14.574555   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:14.598678   927 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 13:14:14.598536   930 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:14.753705  1002 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:14.740628  1003 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:14.754345  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:14.728617   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.388525  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.389345   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.389901  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.389986  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:15.391168  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:14.852674  1596 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:15.392038  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.393195  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.394120  4830 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:15.394013  4547 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:14.562345  1599 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:14.708985  4142 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:15.417289  4142 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.694224  4453 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.420363   996 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:15.422199  4453 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.605783  4386 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.959918  4514 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.424958  4514 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.427074  4514 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.427222  4386 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.521549  4594 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.599990  4653 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.430003  4594 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.430581  4653 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.526122  4657 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:15.431821  4657 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.415390  1599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 659) took 1074 ms. Trace:
I20260709 13:14:15.432549  1599 rpcz_store.cc:276] 0709 13:14:14.340997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.345918 (+  4921us) service_pool.cc:224] Handling call
0709 13:14:15.415302 (+1069384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:15.433869  4657 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.548792  4688 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.734712  4685 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.612921  4694 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:14.664765  4719 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.440016  4688 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.441329  4688 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.932574  4725 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.443120  4725 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.444005  4688 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.334004   896 leader_election.cc:400] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:15.447659   896 leader_election.cc:403] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:15.448386   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 114530us
I20260709 13:14:14.960427  4746 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.450451  4746 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:14.676023  4757 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:14:14.832327  4762 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:15.479317  4762 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:14.563808  4766 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:14.754222  4774 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:14.730255  4780 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.492372  4774 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:14.899271  4783 raft_consensus.cc:3055] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:15.495414  4783 raft_consensus.cc:740] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:15.495937  4766 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.496423  4783 consensus_queue.cc:260] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.497685  4783 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.498534  4766 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:14.668154  4784 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.502386  4784 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.891587  4787 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.503744   801 placement_policy_util.cc:248] tablet 7cf198f3795b43ff951a11167bc21580: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:14:15.503919  4787 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.506052  4787 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:14.941856  4790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.000s	sys 0.011s
W20260709 13:14:14.890301  4791 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.546s	user 0.001s	sys 0.002s
I20260709 13:14:14.594373  4793 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:15.512830   928 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:14.730229  4794 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:14:15.519085   753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 227) took 9662 ms. Trace:
I20260709 13:14:15.023285  4452 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:15.521775  4452 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:15.523466   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1933) took 1477 ms. Trace:
I20260709 13:14:15.520996   753 rpcz_store.cc:276] 0709 13:14:05.860746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.922751 (+ 62005us) service_pool.cc:224] Handling call
0709 13:14:07.224843 (+1302092us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02b47f3a 00007f0d088fe1cc 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc 00007f0d06ebc392 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:15.516647 (+8291804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:13.237641 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:13.238311 (+   670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:13.238361 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:14:13.238385 (+    24us) tablet.cc:662] Decoding operations
0709 13:14:13.239294 (+   909us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:13.239370 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:13.239398 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:14:13.240080 (+   682us) tablet.cc:701] Row locks acquired
0709 13:14:13.240131 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:14:13.240305 (+   174us) write_op.cc:270] Start()
0709 13:14:13.240450 (+   145us) write_op.cc:276] Timestamp: P: 1783602853240244 usec, L: 0
0709 13:14:13.240483 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:14:13.261377 (+ 20894us) log.cc:844] Serialized 2805 byte log entry
0709 13:14:13.627992 (+366615us) op_driver.cc:464] REPLICATION: finished
0709 13:14:13.630489 (+  2497us) write_op.cc:301] APPLY: starting
0709 13:14:13.669709 (+ 39220us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:13.678913 (+  9204us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:13.679103 (+   190us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:13.685444 (+  6341us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:13.685823 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:14:13.685891 (+    68us) write_op.cc:312] APPLY: finished
0709 13:14:13.686288 (+   397us) log.cc:844] Serialized 59 byte log entry
0709 13:14:13.686510 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:13.686657 (+   147us) write_op.cc:454] Released schema lock
0709 13:14:13.686736 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:13.687331 (+   595us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:13.687374 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1270479,"reported_tablets":49,"spinlock_wait_cycles":67316480,"child_traces":[["op",{"apply.queue_time_us":2186,"apply.run_cpu_time_us":4017,"apply.run_wall_time_us":56806,"num_ops":7,"prepare.queue_time_us":156425,"prepare.run_cpu_time_us":42216,"prepare.run_wall_time_us":189289,"raft.queue_time_us":106263,"raft.run_cpu_time_us":1049,"raft.run_wall_time_us":2870,"replication_time_us":387336,"thread_start_us":298227,"threads_started":4,"wal-append.queue_time_us":166923,"wal-append.run_cpu_time_us":5260,"wal-append.run_wall_time_us":1257935}]]}
I20260709 13:14:15.523960   995 rpcz_store.cc:276] 0709 13:14:14.045728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.213897 (+168169us) service_pool.cc:224] Handling call
0709 13:14:14.214265 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.214715 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.214749 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.215242 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.505098 (+289856us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:14.700155 (+195057us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.708398 (+  8243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.708495 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.522929 (+814434us) raft_consensus.cc:1685] finished
0709 13:14:15.522950 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.523342 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183377,"prepare.queue_time_us":176200,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":8126,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":8193,"thread_start_us":308336,"threads_started":2,"wal-append.queue_time_us":315066}
W20260709 13:14:15.077111   851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.006s
I20260709 13:14:15.178462  4615 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.526952   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2239) took 1090 ms. Trace:
I20260709 13:14:15.526670  4615 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:15.180820   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.527343   851 rpcz_store.cc:276] 0709 13:14:14.436668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.436811 (+   143us) service_pool.cc:224] Handling call
0709 13:14:14.437095 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.518303 (+ 81208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.518336 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.520500 (+  2164us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.526584 (+1006084us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:15.526636 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.526715 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.526762 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:15.526774 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.526899 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30821,"raft.queue_time_us":80121,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5464,"thread_start_us":1022717,"threads_started":2,"wal-append.queue_time_us":546277}
I20260709 13:14:15.530369   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.248562  4800 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.249531  4826 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:15.530819   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 554) took 1064 ms. Trace:
W20260709 13:14:15.268682   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 308987us
I20260709 13:14:15.269436  4626 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:15.279956  4692 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:15.533794  4626 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:15.534309  4626 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.533978  4826 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.280934   847 raft_consensus.cc:2468] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:15.535497  4615 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.281327  4494 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:15.535921   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2249) took 1042 ms. Trace:
I20260709 13:14:15.279817  4771 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:15.536176  4826 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:15.282851  4393 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:15.536988  4393 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.537357  4393 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.538265   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.539445  4837 consensus_queue.cc:1059] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:15.540139   848 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:15.285080   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:15.543960   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 258952us
I20260709 13:14:15.543872  4800 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.545439  4793 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:15.545919  4793 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:15.546365  4793 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.285805  4726 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.286370  4815 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:15.291056  4814 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:15.304117   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.547179  4726 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.547770  4814 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.548583   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.549197   998 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.549358  4726 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.549762  4386 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:15.308956  4501 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:15.316468  1000 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:14:15.318971  4806 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.641s	user 0.000s	sys 0.005s
I20260709 13:14:15.319929  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.565531  1001 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:15.565748   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 110585us
I20260709 13:14:15.566468  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.320221   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.324440  4832 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.325809   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:15.331267  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:15.334550  4831 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:15.336231   852 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:15.569321   852 pending_rounds.cc:85] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 4
I20260709 13:14:15.569370  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.569737   852 pending_rounds.cc:107] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:15.569751   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.570147   852 raft_consensus.cc:2889] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:15.336470   751 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.570628  1595 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:15.566480   896 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:15.373070  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.393982  1004 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:15.382405  4459 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:14:15.574208  4837 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.397202  4631 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:14:14.568077  4573 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:15.439563  4685 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.441229  4719 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.684443  4719 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:14.607652  4720 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:15.686842  4720 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:15.440420  4694 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.445667  4725 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:14.614434  4736 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.691318  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:15.447345   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.692075   849 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:15.693043   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.694298   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.694664  4736 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:15.449275  4653 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:14.576819  4754 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:15.702759  4754 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:15.454591   855 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:15.468564  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 76466.2us
I20260709 13:14:15.471957  4688 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:15.468772  4746 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.490401  4762 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.494899  4774 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.709664   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1921) took 1182 ms. Trace:
I20260709 13:14:15.710053   855 rpcz_store.cc:276] 0709 13:14:14.527171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.818542 (+291371us) service_pool.cc:224] Handling call
0709 13:14:15.709477 (+890935us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000c9ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:15.709608 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432145664}
I20260709 13:14:15.509107  4142 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:15.512349   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2006) took 1502 ms. Trace:
W20260709 13:14:15.509198  4790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.171s	user 0.003s	sys 0.011s
I20260709 13:14:15.519470   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 7cf198f3795b43ff951a11167bc21580 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42376
W20260709 13:14:15.238277   850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.007s	sys 0.031s
I20260709 13:14:15.532114   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:15.532907   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:15.714175   850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.194s	user 0.007s	sys 0.031s
I20260709 13:14:15.533368   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.699314  4736 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.714515   851 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:15.546643  4800 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:15.741212  1597 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:15.740870  4736 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.742623  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:15.745707  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:15.531349   928 rpcz_store.cc:276] 0709 13:14:14.466611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.486388 (+ 19777us) service_pool.cc:224] Handling call
0709 13:14:14.486729 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.513239 (+1026510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.530643 (+ 17404us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400319a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:15.530776 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2359319296}
I20260709 13:14:15.748116   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.715739   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.749811   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.554719   995 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:15.556192   854 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:14:15.550324  4386 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:15.330447  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:15.399642  4767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.012s	sys 0.001s
I20260709 13:14:15.571525   751 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 8 to 10, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 10 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.711792  4573 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:15.581816   927 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:15.580734   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:15.549876  4814 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.579123  4692 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:15.586107  4842 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:15.586370   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.588222   930 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:14:15.522771  4452 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.590325   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:15.595075   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:15.579170  4826 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:15.602331   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:15.582099  4657 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:15.569813  4836 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:14:15.576537  4832 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.759644   751 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:15.582314  4732 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:15.607654   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.582391  4784 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:15.613189   924 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:15.612960   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.771474   925 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:15.621352   998 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
W20260709 13:14:15.755828  4767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.410s	user 0.012s	sys 0.001s
I20260709 13:14:15.773833   998 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:14:15.636227   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58976.8us
I20260709 13:14:15.647271  1592 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:14:15.651041  4743 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.651077   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:15.776676   997 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:15.777383  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:15.652100  4793 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.654830  1002 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:15.658810  4626 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.661875  4393 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.769852  1598 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:15.664578  4815 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:15.582465  4837 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.665874   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1916) took 1217 ms. Trace:
I20260709 13:14:15.668676  4726 raft_consensus.cc:993] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:15.667469   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:15.677568   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.689704  4694 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:15.690836  1000 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:15.697539  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.697582   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.699021  4719 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.713660  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:15.536312   847 rpcz_store.cc:276] 0709 13:14:14.493707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.835954 (+342247us) service_pool.cc:224] Handling call
0709 13:14:15.535861 (+699907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:15.730615  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:15.860123   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.861110   853 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
W20260709 13:14:15.714844   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 182780us
I20260709 13:14:15.733017   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.733583   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1909) took 1325 ms. Trace:
W20260709 13:14:15.864456  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 166967us
I20260709 13:14:15.731271  4688 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.712146   926 rpcz_store.cc:276] 0709 13:14:14.010238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.010402 (+   164us) service_pool.cc:224] Handling call
0709 13:14:14.019754 (+  9352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:14.031930 (+ 12176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.031973 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:14.060728 (+ 28755us) log.cc:844] Serialized 52 byte log entry
0709 13:14:14.191220 (+130492us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:14.251036 (+ 59816us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.258026 (+  6990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.258096 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.889635 (+631539us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:15.511848 (+622213us) raft_consensus.cc:1685] finished
0709 13:14:15.512102 (+   254us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.512281 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1366,"prepare.queue_time_us":61153,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":60684,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":5475,"spinlock_wait_cycles":70297088,"thread_start_us":77212,"threads_started":1,"wal-append.queue_time_us":233022}
I20260709 13:14:15.734789   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.735956  4754 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.739059  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:15.729776  4720 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.751578   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.758467   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.874446   846 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:15.760932   921 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:15.752076   854 pending_rounds.cc:85] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 4
W20260709 13:14:15.758041   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 177359us
I20260709 13:14:15.764165   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.881119   854 pending_rounds.cc:107] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:14:15.881500   930 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:15.881727   929 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:15.881531   854 raft_consensus.cc:2889] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:15.579275  4831 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.753727   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:15.768306   999 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:15.883888   854 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:15.884732   855 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.885111   853 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:15.885689   853 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:15.783957   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:15.833556   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.762146  4826 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:15.887013   928 raft_consensus.cc:2393] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:15.888417   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 104602us
I20260709 13:14:15.889343   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:15.755717  4142 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.765452  4657 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:15.837862  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60549.7us
I20260709 13:14:15.840708  1001 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.764806  4700 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:15.847946   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.894033   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.781522  4847 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:14:15.854314   849 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:15.924510   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:15.924965   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79090.5us
I20260709 13:14:15.925208   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.892853   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:15.929617   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.930812   821 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.874048  4335 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:15.782063  4784 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.846290   852 rpcz_store.cc:276] 0709 13:14:14.448002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.450501 (+  2499us) service_pool.cc:224] Handling call
0709 13:14:14.450774 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.578884 (+1128110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.665551 (+ 86667us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b540041f5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:15.665788 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025056640}
I20260709 13:14:15.838042  4725 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:15.869472   923 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:15.840885  4743 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.868100  4736 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:15.782500  4452 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:15.874475   927 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:15.765991   751 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.757566  4573 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.781675   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.915567  4790 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:15.772248  4771 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.097s
W20260709 13:14:15.895099   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 61618.1us
I20260709 13:14:15.839772  4815 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:15.758518  4692 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:15.757742  4746 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:15.854683  1002 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:15.933310   849 raft_consensus.cc:1240] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:16.011933  1601 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:15.935256   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.863250   850 rpcz_store.cc:276] 0709 13:14:14.407710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.407863 (+   153us) service_pool.cc:224] Handling call
0709 13:14:14.408200 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:14.474373 (+ 66173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.474402 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:14.488029 (+ 13627us) log.cc:844] Serialized 52 byte log entry
0709 13:14:15.318286 (+830257us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:15.714621 (+396335us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:15.732809 (+ 18188us) log.cc:844] Serialized 14 byte log entry
0709 13:14:15.733086 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.733166 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.733213 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:15.733352 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.733505 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2514,"prepare.queue_time_us":28945,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":84210,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":8038,"spinlock_wait_cycles":13169536,"thread_start_us":1259908,"threads_started":2,"wal-append.queue_time_us":175746}
I20260709 13:14:15.938144   921 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:15.927304  4766 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:15.893173  4826 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:15.976073   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:15.977627   751 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 5 to 8, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 8 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.937865  4335 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.017167   751 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 6 to 8, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 8 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.887358   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:15.893200  4800 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.984630  1001 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:15.993976   922 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:15.893311  4631 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:14:15.912783  4615 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:14:15.874564  4719 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:15.990700  4514 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.975412  4780 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:15.943723  4783 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:15.941602  4767 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.020325   924 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:15.986475  4685 raft_consensus.cc:1081] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:15.942648  4725 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:16.003986  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 89449.5us
I20260709 13:14:15.984655  4842 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:14:16.034322   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800098260 after lost signal to thread 32227
I20260709 13:14:16.034377  4854 tablet_replica.cc:333] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:14:16.035101  4800 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:15.991178  4547 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:15.905918  4594 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:14:16.098433   997 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:14:16.122422   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 193527us
I20260709 13:14:16.005162  4790 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:16.104436  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 113269us
W20260709 13:14:16.105280   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 216012us
I20260709 13:14:16.130885   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:16.030855  4335 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:16.108771   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 173563us
I20260709 13:14:16.135836   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:15.943284  4706 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.150s
I20260709 13:14:16.137645   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.138484   927 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:16.142478  4826 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.143882  4594 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.144552   847 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:16.118604   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800363340 after lost signal to thread 787
W20260709 13:14:16.146348   876 debug-util.cc:398] Leaking SignalData structure 0x7b08003e0c00 after lost signal to thread 879
W20260709 13:14:16.148022   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800380de0 after lost signal to thread 954
W20260709 13:14:16.148358   876 debug-util.cc:398] Leaking SignalData structure 0x7b08002a8a60 after lost signal to thread 955
I20260709 13:14:16.139400   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:16.150203   876 debug-util.cc:398] Leaking SignalData structure 0x7b0800354360 after lost signal to thread 1028
W20260709 13:14:16.151646   876 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1200 after lost signal to thread 1625
W20260709 13:14:16.151999   876 debug-util.cc:398] Leaking SignalData structure 0x7b08003b10c0 after lost signal to thread 1626
W20260709 13:14:16.152824   876 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0f40 after lost signal to thread 4545
W20260709 13:14:16.153427   876 debug-util.cc:398] Leaking SignalData structure 0x7b08003f3fc0 after lost signal to thread 4709
I20260709 13:14:16.110903  4814 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:16.120281  4725 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.183732   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.204376  4393 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.205031  4826 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.206049   927 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:16.206564   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70797.5us
I20260709 13:14:16.108563  4514 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.144338  4594 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.214046   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.215098   927 raft_consensus.cc:2368] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11.
I20260709 13:14:16.119922  4767 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:16.223734   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 92910.7us
I20260709 13:14:16.222110   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.227861   927 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.210041  4845 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.209795  4783 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:14:16.219579  4853 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:16.217422  4594 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.213366  4826 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:16.146409  4815 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.205967  4814 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:16.283978   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 135206us
I20260709 13:14:16.219328  4746 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:16.153810  4692 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.291646   927 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:16.244606  4626 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.230724  4700 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:16.275884  4783 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:16.324541  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:16.334057  1599 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:16.330562   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.295367 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:16.284993  4872 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.336625  4872 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:16.336774  4743 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.337044  4872 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.337842   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:16.338660   927 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:16.336808  4700 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.346697   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.347427   927 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.285184  4790 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.350752   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.335912   855 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:16.285447  4826 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:16.284428  4780 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:16.364651   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 139425us
I20260709 13:14:16.289702  4688 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.292867  4657 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.326211  4815 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:16.335222  4848 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:14:16.273749  4845 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.369241  4826 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.358896  4142 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.378175  4780 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.379163  4783 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.410682  1592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 409) took 1075 ms. Trace:
I20260709 13:14:16.384676   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.386760  4837 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.394547  4547 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.417155  4547 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.395813  4860 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.387720   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.421288   929 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:16.390139   921 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:16.368484  4868 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 13:14:16.391883   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54147.8us
I20260709 13:14:16.392498  4869 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:16.392792  4814 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.368358  4847 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:16.335522  4784 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.396086   927 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:16.411140  1592 rpcz_store.cc:276] 0709 13:14:15.330383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.331697 (+  1314us) service_pool.cc:224] Handling call
0709 13:14:15.332014 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.646991 (+314977us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400386ba0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b411c0 00007f0d02b42231 00007f0d02b3ed8f 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c
0709 13:14:15.776062 (+129071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.402884 (+626822us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400386ba0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02ab6dc0 00007f0d02b42a02 00007f0d02b3edd6 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:14:16.406005 (+  3121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681274880}
W20260709 13:14:16.422785   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 232238us
I20260709 13:14:16.396365  4763 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:16.392974  4451 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:14:16.433732  4767 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.434615  4514 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.427259  4869 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:16.439728   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:16.419417  4719 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.443878  4879 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:14:16.444850  4800 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.393347  4746 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.446354   995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1941) took 1089 ms. Trace:
I20260709 13:14:16.462922   995 rpcz_store.cc:276] 0709 13:14:15.357151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.759664 (+402513us) service_pool.cc:224] Handling call
0709 13:14:15.759977 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.079053 (+319076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.079102 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.100850 (+ 21748us) log.cc:844] Serialized 26 byte log entry
0709 13:14:16.265114 (+164264us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:16.265163 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.445966 (+180803us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540035b954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:16.446016 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.446068 (+    52us) raft_consensus.cc:1685] finished
0709 13:14:16.446084 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.446293 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82418,"raft.queue_time_us":98455,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12056,"spinlock_wait_cycles":297209728,"thread_start_us":309998,"threads_started":2,"wal-append.queue_time_us":219036}
I20260709 13:14:16.465515   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:16.450160  4862 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:14:16.429096  4875 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:16.457883  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:16.458072   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:16.418365  4860 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.474732  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:16.471083   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.436136  4763 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:16.448838  4872 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.475404  1592 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:16.477360   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:16.452842  4869 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.477996   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.482028   818 leader_election.cc:400] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11.
I20260709 13:14:16.466954  4514 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:16.483410   818 leader_election.cc:403] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:16.460886  4547 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.483278   995 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:16.459906  4725 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.543756   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:16.555339  4885 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:14:16.556174  4386 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.558490  4845 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.552176   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:16.552968   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71675.7us
I20260709 13:14:16.468272  4767 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:16.566061   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:16.481267  4780 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:16.575187  4780 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:16.575716  4780 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.576081  4780 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.576371  4867 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:16.552347   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.577975   995 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:16.576923   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:16.576761  4865 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:16.478974  4875 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.579507   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:16.545600  4864 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.583456  4875 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.583272  4864 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.586615  4864 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:16.589745   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.569399  4451 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.563779  4879 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11.
I20260709 13:14:16.484143  4800 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:16.592036   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.595715   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:16.600708  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:16.601418  1002 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:16.602192  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.602826  1002 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:16.602569   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.603438   997 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:16.604616   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.606559  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1969) took 1174 ms. Trace:
I20260709 13:14:16.605894  4763 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.606936  1000 rpcz_store.cc:276] 0709 13:14:15.432002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.569121 (+137119us) service_pool.cc:224] Handling call
0709 13:14:16.606391 (+1037270us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000eb854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:16.606488 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277490560}
I20260709 13:14:16.591302  4451 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.599035   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.592262  4790 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:16.607498  4763 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.616182  4790 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:16.617179  4790 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.608161  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.613304   927 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:16.615370   854 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:16.626000   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.626235   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.578500  4547 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:16.636355   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.603552  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:16.579632  4767 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.641968   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.642242  1002 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
W20260709 13:14:16.610809  4452 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.622s	user 0.003s	sys 0.007s
I20260709 13:14:16.643312  4547 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.719728  4547 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.609349  4725 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.636157  4864 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:16.723934  4864 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.724371  4864 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.725000  4854 raft_consensus.cc:2243] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:16.725947  4854 raft_consensus.cc:2272] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:16.611317  4872 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.609217  4826 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.730377  4826 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.731315   851 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:16.701350   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1961) took 1379 ms. Trace:
I20260709 13:14:16.734357  4854 ts_tablet_manager.cc:1916] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:16.732751   996 rpcz_store.cc:276] 0709 13:14:15.321512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.421267 (+ 99755us) service_pool.cc:224] Handling call
0709 13:14:15.421556 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.550210 (+128654us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:15.685341 (+135131us) log.cc:844] Serialized 14 byte log entry
0709 13:14:15.893770 (+208429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.893831 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:15.893873 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.143916 (+250043us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180039a840. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:16.701113 (+557197us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400087fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:16.701168 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.701289 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26329,"prepare.queue_time_us":51038,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":132484,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":14278,"spinlock_wait_cycles":320651392,"thread_start_us":335291,"threads_started":3,"wal-append.queue_time_us":299293}
I20260709 13:14:16.645080  4720 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:14:16.686086  4594 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.701445  4767 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.705986  4784 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:16.712968   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60284.5us
I20260709 13:14:16.743009   896 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:16.609529  4451 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:16.744177   896 raft_consensus.cc:2686] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:14:16.750697  4767 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.750667   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:16.746760  4335 consensus_meta.cc:287] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.001s
I20260709 13:14:16.766407   998 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1944) took 1407 ms. Trace:
I20260709 13:14:16.759640   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:16.751483  4547 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.764230   853 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.773201   853 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:16.770695   998 rpcz_store.cc:276] 0709 13:14:15.358561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.774894 (+416333us) service_pool.cc:224] Handling call
0709 13:14:15.775268 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.775555 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:15.854506 (+ 78951us) log.cc:844] Serialized 14 byte log entry
0709 13:14:16.615903 (+761397us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b1800375ca0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df23a 00007f0d02b49a2a
0709 13:14:16.765147 (+149244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:16.765225 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:16.765284 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.766148 (+   864us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.766331 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4762,"prepare.queue_time_us":72124,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":13524,"raft.queue_time_us":311,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2248,"spinlock_wait_cycles":10060032,"thread_start_us":148291,"threads_started":1}
W20260709 13:14:16.774564   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.750200  4784 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.759749  4706 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:16.751479  4767 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.784631  4864 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.786262   853 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:16.788290   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:16.799350   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2258) took 1243 ms. Trace:
I20260709 13:14:16.800040   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:16.793332   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.763711  4335 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.798782   996 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:16.805037   925 raft_consensus.cc:2393] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:16.796757  4594 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.803881   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.799883   853 rpcz_store.cc:276] 0709 13:14:15.555761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.884661 (+328900us) service_pool.cc:224] Handling call
0709 13:14:15.884968 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:16.762306 (+877338us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b4c0013f7c0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d00e0ba20 00007f0d00e1ca29 00007f0d00de2644 00007f0d00e09f2d 00007f0d00e09b93 00007f0d02b3a18b 00007f0d02b3acb5 00007f0d02b35f30 00007f0d02b377de 00007f0d02b41b49 00007f0d02b42416
0709 13:14:16.799125 (+ 36819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.799303 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67586816}
I20260709 13:14:16.765977  4859 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:16.775218  4766 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:16.808336   923 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 13:14:16.810688  1003 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:16.808732  4452 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:16.817417  4891 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:14:16.819087  4892 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:14:16.803407  4894 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:16.819365  4784 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.833431  4864 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.838347  4780 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.825770   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:16.845597   855 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.851341  4869 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.854730  4869 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.819947  4894 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.807540  4766 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.855837  4766 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.856980  4547 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.860083  1000 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:16.863339  4452 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:16.864267  4452 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.866068   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.866621  4894 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.868625  4767 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.873831  4767 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:14:16.875368  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.876644   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.877537  4780 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.824146  4335 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.881330   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:16.881978   927 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.890590  4793 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:16.894658  4832 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.902350   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 572) took 1195 ms. Trace:
I20260709 13:14:16.902829   922 rpcz_store.cc:276] 0709 13:14:15.706783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.765160 (+ 58377us) service_pool.cc:224] Handling call
0709 13:14:16.902116 (+1136956us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54003caad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:16.902257 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24005376}
I20260709 13:14:16.910058  4386 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:16.913908  4874 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:14:16.915328  1601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 648) took 1224 ms. Trace:
I20260709 13:14:16.915756  1601 rpcz_store.cc:276] 0709 13:14:15.690725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.691058 (+   333us) service_pool.cc:224] Handling call
0709 13:14:16.915160 (+1224102us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400091d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:16.915270 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735651200}
W20260709 13:14:16.918346   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.918604  4452 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:16.914067   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.921358  4832 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:16.921540   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48574.4us
I20260709 13:14:16.921954  4832 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:16.922390  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60392.1us
I20260709 13:14:16.924582  4832 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:16.925614   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:16.926743  4867 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:16.926939  4688 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.927182  4867 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:16.927393  4688 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.927799  4688 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.928104  4867 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.929302  4867 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:16.931550  4832 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.932801  4793 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:16.936039   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:16.937211  4793 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:16.937659  4793 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:16.936866  4832 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.937863   851 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:16.938524  4793 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.939801  4793 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:16.940292  4887 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.359s	user 0.000s	sys 0.003s
I20260709 13:14:16.942113   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:16.942677   850 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.936319  4732 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.943888  4732 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:16.944918  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:16.945498  1596 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.945719   930 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:16.947014  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:16.947590  1599 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.951627  1601 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:16.954641  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 1970) took 1522 ms. Trace:
I20260709 13:14:16.955138  1001 rpcz_store.cc:276] 0709 13:14:15.432302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.566236 (+133934us) service_pool.cc:224] Handling call
0709 13:14:16.954433 (+1388197us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54003e2954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:16.954571 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628935424}
I20260709 13:14:16.958345  4573 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.958849  4863 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:16.960319  4501 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:16.960786  4501 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:16.962276  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1967) took 1639 ms. Trace:
I20260709 13:14:16.961922  4501 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:16.962647  1004 rpcz_store.cc:276] 0709 13:14:15.323305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.674026 (+350721us) service_pool.cc:224] Handling call
0709 13:14:15.674342 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.975404 (+301062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.975438 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.975482 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:15.975513 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.962055 (+986542us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540035aa54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:16.962103 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.962222 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691795200}
I20260709 13:14:16.971215  4831 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:16.974481   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.975333   923 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:16.975464  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 1960) took 1654 ms. Trace:
I20260709 13:14:16.975695  1000 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:16.975903  1003 rpcz_store.cc:276] 0709 13:14:15.321212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.394889 (+ 73677us) service_pool.cc:224] Handling call
0709 13:14:15.395197 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.811164 (+1415967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.975238 (+164074us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54000ea954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:16.975407 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3254632192}
I20260709 13:14:16.976604   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:16.977099  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:16.981760  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.993716  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:16.994104  4501 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.918358   855 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:16.998708  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.001008  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.002180  1000 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:17.004319  4874 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:17.004495  4826 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:17.004884  4874 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.005088  4826 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.006175   927 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:17.006990  4874 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.007236   847 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:17.007149  4826 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.008560  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.008793  4831 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:17.009320  1601 raft_consensus.cc:2393] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:17.009662  4859 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:17.010017  4859 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:17.009966   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.011128  4859 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.011729   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.012465  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:17.013661  4863 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:17.014123  4863 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.014252   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.014575  4863 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.016427  4831 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:17.016927  4831 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.017333  4831 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:17.015250  4859 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:17.020236   926 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:17.022612   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.023228   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.023941   852 raft_consensus.cc:2368] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
W20260709 13:14:17.023806   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 75339.4us
I20260709 13:14:17.024470   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.024755   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:17.025238   846 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:17.026754   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.029096  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.029810  1001 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:17.033739   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.034485   998 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:17.036854   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.038306   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.040458   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.040530   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.041509  4832 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:17.042181  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.043471   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.027439  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.024503  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.054934   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.056334   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.056929  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.058383  1599 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:17.060374  1566 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.061299   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.061457  4547 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:17.061573  4763 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.062083  4547 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:17.062228  4763 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.063195  4830 raft_consensus.cc:2804] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:17.063719  4830 raft_consensus.cc:697] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:17.063791  4869 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:17.064837  4763 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.063019  4547 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.065100  4830 consensus_queue.cc:237] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.060417   996 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:17.063021   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.069813   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.069903   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:17.071867   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.073041  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.073707  1000 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:17.058450   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.057258   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.077441   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.078356   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:17.078755   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.079857   930 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
I20260709 13:14:17.080140   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.080276   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:17.081043   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.082005   930 raft_consensus.cc:2368] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:14:17.083405   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:17.088728   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.089357   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.092176   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.078806  1000 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:17.089426   849 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:17.105659  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.106348  1004 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.106773  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.107507  1002 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:17.108116   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:17.108884   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.087509   998 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
W20260709 13:14:17.110528   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.110646   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.112073   855 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:17.113857  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.049053  4783 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:17.114627   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.117405  1601 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:17.136058   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:17.038838  4874 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:17.136485  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.141531  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.141755  1599 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:17.052271  4826 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:17.137143   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 56005.4us
I20260709 13:14:17.137463   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:17.161657   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.174568  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.088950  4815 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:17.115274  4879 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:17.056087   852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:17.184152   852 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:17.184543   852 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.149899  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:17.144556   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:17.177445   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56076.2us
I20260709 13:14:17.119517   850 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:17.088434  4864 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:17.139335   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:17.181444  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.182367   926 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:17.178347   930 raft_consensus.cc:3060] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:17.186816   849 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:17.187116   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.115510  4706 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:17.191103   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51859.9us
I20260709 13:14:17.202913   821 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:17.070925  4766 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.172345  4881 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:17.114224  4875 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:14:17.144362  4700 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:17.209200  1601 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:14:17.218736   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:17.208493  4706 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.140367  4494 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:14:17.202574  4815 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:17.220546  4826 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:17.221244   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:17.221792  4766 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.222244  4881 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.250383  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:17.268642   997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.071s	sys 0.052s
I20260709 13:14:17.173197  4874 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:17.275197  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.280318   930 raft_consensus.cc:2468] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:17.136969   855 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:17.224704  4732 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:17.352258   997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.071s	sys 0.052s
I20260709 13:14:17.248021  4700 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.254133  4860 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:17.437880   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 216678us
I20260709 13:14:17.484524   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:17.263695  4832 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.437225   921 raft_consensus.cc:2410] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:17.355631  4881 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:17.202776  4864 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.445644  4594 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:17.437564   930 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:17.255578  4688 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.474045   847 raft_consensus.cc:2410] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:14:17.455752  4896 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.135s	user 0.008s	sys 0.000s
W20260709 13:14:17.493834   999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.000s	sys 0.046s
W20260709 13:14:17.508136   999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.000s	sys 0.046s
I20260709 13:14:17.452299  4657 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.476253  4700 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.516636   855 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:17.519752  4815 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.522483   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:17.522723  4386 consensus_meta.cc:287] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.005s
I20260709 13:14:17.523113  4657 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.526808  1000 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:17.526724  4700 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.527456  4657 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:17.528131   855 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
W20260709 13:14:17.532043  4894 consensus_meta.cc:287] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.007s
I20260709 13:14:17.532584  4894 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.535007  4894 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.535048   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.529256  4700 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.525183  4864 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.565426   855 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:14:17.566159  1596 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.620s	user 0.015s	sys 0.003s
I20260709 13:14:17.568104  1596 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:17.568931  1595 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:17.572817   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.492084  4784 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.580269  4784 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:17.558419   852 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:17.503043  4617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.006s	sys 0.009s
I20260709 13:14:17.586964   852 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.583403  4790 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:17.588894   852 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:17.592435   852 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:17.593824   852 raft_consensus.cc:1240] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 13:14:17.593911  4831 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:14:17.594576  4831 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:17.595407  4863 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.000s
W20260709 13:14:17.595973  4451 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.836s	user 0.012s	sys 0.001s
I20260709 13:14:17.596601  4573 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:17.603071  4859 consensus_meta.cc:287] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
I20260709 13:14:17.604034  4706 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:17.584100  4617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.007s	sys 0.010s
I20260709 13:14:17.540019  4754 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:17.607867  1004 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:17.549619  4688 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:17.604444  4854 ts_tablet_manager.cc:1929] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 8.6
W20260709 13:14:17.649768  1002 consensus_meta.cc:287] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:14:17.691736  1002 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:17.652798   923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.019s
W20260709 13:14:17.653455   851 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.015s	sys 0.028s
W20260709 13:14:17.693007   923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.007s	sys 0.020s
W20260709 13:14:17.693089   851 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.015s	sys 0.028s
W20260709 13:14:17.706741  4889 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 1.141s	user 0.007s	sys 0.000s
I20260709 13:14:17.708451  4854 log.cc:1199] T 7cf198f3795b43ff951a11167bc21580 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/7cf198f3795b43ff951a11167bc21580
I20260709 13:14:17.547693  4594 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:17.652421  4867 consensus_meta.cc:287] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.001s
I20260709 13:14:17.547770  4881 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:17.718098  1001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.039s	sys 0.042s
W20260709 13:14:17.719444  1001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.040s	sys 0.042s
W20260709 13:14:17.721210   895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.009s	sys 0.022s
W20260709 13:14:17.721730   895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.009s	sys 0.022s
W20260709 13:14:17.716089  4746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.014s	sys 0.037s
I20260709 13:14:17.650992  4762 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.506009  4832 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:17.716037  4830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.023s
W20260709 13:14:17.722502   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 639159us
W20260709 13:14:17.725189  4501 consensus_meta.cc:287] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.005s
I20260709 13:14:17.747798   927 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:17.756364   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:17.733072   853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.012s	sys 0.009s
W20260709 13:14:17.757709   853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.012s	sys 0.010s
W20260709 13:14:17.732690   928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.009s	sys 0.012s
I20260709 13:14:17.652093  4863 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.652200  4451 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.653859  4573 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:17.733718   924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.006s	sys 0.014s
W20260709 13:14:17.773676   924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.006s	sys 0.014s
W20260709 13:14:17.758553   928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.009s	sys 0.012s
I20260709 13:14:17.721380  4874 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:17.726042  4746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.014s	sys 0.038s
W20260709 13:14:17.749456  4626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.015s	sys 0.005s
W20260709 13:14:17.746783  4692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.017s	sys 0.015s
W20260709 13:14:17.752372  4830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.007s	sys 0.024s
W20260709 13:14:17.747008  4547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.011s	sys 0.020s
I20260709 13:14:17.768884  4854 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42376 (request call id 75) took 2254 ms. Trace:
I20260709 13:14:17.778887  4787 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:17.783100  4692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.017s	sys 0.015s
I20260709 13:14:17.784390   922 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:17.783376   818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.030s
W20260709 13:14:17.780797  4626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.016s	sys 0.005s
I20260709 13:14:17.782702  4706 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:17.750886  4762 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:17.792654  1599 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
W20260709 13:14:17.786077  4547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.012s	sys 0.020s
W20260709 13:14:17.798641   896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.036s	sys 0.040s
W20260709 13:14:17.789944   818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.030s
W20260709 13:14:17.798877   854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.011s	sys 0.026s
W20260709 13:14:17.800967   854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.011s	sys 0.026s
W20260709 13:14:17.796633  1564 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.006s	sys 0.018s
W20260709 13:14:17.801447  1564 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.006s	sys 0.019s
W20260709 13:14:17.801920  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 595702us
W20260709 13:14:17.804265  1003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.003s	sys 0.018s
W20260709 13:14:17.798980   896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.036s	sys 0.040s
I20260709 13:14:17.807365  4832 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:17.806150  1003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.003s	sys 0.018s
I20260709 13:14:17.808842  4864 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:17.811363   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:17.812822   821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.011s	sys 0.014s
W20260709 13:14:17.813237   821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.011s	sys 0.014s
W20260709 13:14:17.813884   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 611015us
I20260709 13:14:17.814723   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:17.785220   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:17.826992   894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.000s	sys 0.038s
W20260709 13:14:17.829350   894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.000s	sys 0.039s
W20260709 13:14:17.838728  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.844074  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.848232  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:17.857671   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 72533.8us
I20260709 13:14:17.830446   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.861987  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:17.862417   821 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:17.875690   970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.024s	sys 0.037s
I20260709 13:14:17.785925  4854 rpcz_store.cc:276] 0709 13:14:15.516172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.516339 (+   167us) service_pool.cc:224] Handling call
0709 13:14:16.033862 (+517523us) ts_tablet_manager.cc:1151] Deleting tablet 7cf198f3795b43ff951a11167bc21580
0709 13:14:16.033992 (+   130us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:14:16.930379 (+896387us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:17.584048 (+653669us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:17.768762 (+184714us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":589251,"tablet-delete.queue_time_us":304051,"thread_start_us":120296,"threads_started":1}
I20260709 13:14:17.856982  4874 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.856901  4746 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:17.896154  4451 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:17.896219  4881 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.896158  4863 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.866472  4706 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:17.898689  4594 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:17.899258  4783 consensus_meta.cc:287] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.001s
I20260709 13:14:17.906457   846 raft_consensus.cc:2410] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:14:17.906955   821 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:17.910312   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 498) took 2130 ms. Trace:
W20260709 13:14:17.880338   970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.024s	sys 0.039s
I20260709 13:14:17.863662  4864 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:17.865937  4501 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:17.865408  4784 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:17.910808   997 rpcz_store.cc:276] 0709 13:14:15.783449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.613695 (+830246us) service_pool.cc:224] Handling call
0709 13:14:16.614068 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.627874 (+ 13806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.627920 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.643551 (+ 15631us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.464515 (+820964us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:17.508122 (+ 43607us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.909906 (+401784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.910013 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.910055 (+    42us) raft_consensus.cc:1685] finished
0709 13:14:17.910068 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.910238 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43060,"prepare.queue_time_us":43424,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":155514,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":11699,"thread_start_us":1153923,"threads_started":2,"wal-append.queue_time_us":1181902}
I20260709 13:14:17.910181  4872 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:17.866107  4867 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:17.894261   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1948) took 2510 ms. Trace:
I20260709 13:14:17.926947   999 rpcz_store.cc:276] 0709 13:14:15.383513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.891851 (+508338us) service_pool.cc:224] Handling call
0709 13:14:15.892210 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:16.864615 (+972405us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b1800375a60. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0d026e74bc 00007f0cffa3548b 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa4f455 00007f0d02b37060 00007f0d02b37bfb 00007f0d02b4264b
0709 13:14:16.864750 (+   135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.864776 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:16.865364 (+   588us) log.cc:844] Serialized 52 byte log entry
0709 13:14:17.508718 (+643354us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:17.508787 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.508871 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.893879 (+385008us) raft_consensus.cc:1685] finished
0709 13:14:17.893899 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.894191 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70549,"raft.queue_time_us":135401,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":50554,"spinlock_wait_cycles":8918912,"thread_start_us":806618,"threads_started":2,"wal-append.queue_time_us":936920}
W20260709 13:14:17.916102   929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.011s	sys 0.016s
W20260709 13:14:17.928711   929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.011s	sys 0.016s
W20260709 13:14:17.906299   895 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:17.916669   967 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.001s	sys 0.024s
W20260709 13:14:17.918434   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:17.929641   967 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.001s	sys 0.024s
I20260709 13:14:17.865509  4894 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:17.910710  4783 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:17.900741  4874 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:17.930022   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 820433us
I20260709 13:14:17.930685   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.907361  4831 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:17.907570  4451 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:14:17.934118   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.934333   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.935101   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.919323  4494 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.977941   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:17.977458   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:17.978839   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:17.978827   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.915259  4594 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:17.979907  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.994390  1002 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.922412  4706 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.978827   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:17.987579   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:17.984645   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:17.928748  4867 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.998404   997 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:17.933552  4826 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.004973   849 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:17.983268   851 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:17.928287  4617 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.975471  4783 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.993559   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:17.984242  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:17.932098  4864 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:17.978271  4875 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:18.014108   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 84207us
W20260709 13:14:18.014665  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 161056us
I20260709 13:14:17.984099  4784 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.014720   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:17.975525  4915 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:17.980427  4815 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.020896   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.015004   996 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:18.031533  1002 raft_consensus.cc:2468] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:18.018273  4793 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.005038  4867 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:18.047842   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58468.3us
I20260709 13:14:18.050414   895 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:17.932096  4879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.006s	sys 0.012s
I20260709 13:14:18.013448  4831 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.080329   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:18.059158   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.022259  4783 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.022686  4617 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.054414  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.082484  4831 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.048832   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 7cf198f3795b43ff951a11167bc21580 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:14:18.083969   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2052) took 1044 ms. Trace:
I20260709 13:14:18.054191   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:18.082847  4923 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:18.022331  4762 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:18.086259   737 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 149766us
I20260709 13:14:18.018275  4921 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.086169   921 rpcz_store.cc:276] 0709 13:14:17.040030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.040186 (+   156us) service_pool.cc:224] Handling call
0709 13:14:18.083767 (+1043581us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400090454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.083914 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647997312}
I20260709 13:14:18.000229  4494 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.011721  4706 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.013622  4826 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:18.078971  4879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.006s	sys 0.018s
I20260709 13:14:18.049721  4931 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.049203  4784 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.136947  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 503) took 1038 ms. Trace:
I20260709 13:14:18.136271  4931 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.137358  1003 rpcz_store.cc:276] 0709 13:14:17.098938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.113458 (+ 14520us) service_pool.cc:224] Handling call
0709 13:14:17.113800 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:17.114191 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.114220 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:17.147874 (+ 33654us) log.cc:844] Serialized 52 byte log entry
0709 13:14:17.809813 (+661939us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:17.811858 (+  2045us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.813510 (+  1652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.813616 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.136722 (+323106us) raft_consensus.cc:1685] finished
0709 13:14:18.136746 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.136888 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4101,"prepare.queue_time_us":1274,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1070,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":73399,"thread_start_us":582430,"threads_started":1,"wal-append.queue_time_us":732098}
I20260709 13:14:18.140085  4784 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.140717  4831 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.141346  4784 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:18.145707  4905 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
W20260709 13:14:18.147220  4922 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:14:18.148339   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 588) took 1097 ms. Trace:
I20260709 13:14:18.152031   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 582) took 1338 ms. Trace:
I20260709 13:14:18.152472   853 rpcz_store.cc:276] 0709 13:14:16.814170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.814366 (+   196us) service_pool.cc:224] Handling call
0709 13:14:16.814793 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.815240 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.815285 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.815909 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.758276 (+942367us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:17.758451 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.758559 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.151794 (+393235us) raft_consensus.cc:1685] finished
0709 13:14:18.151814 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.151968 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1400,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":941862,"threads_started":1,"wal-append.queue_time_us":1065519}
W20260709 13:14:18.108141   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 100430us
I20260709 13:14:18.129696  4494 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.130617  4927 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:18.131973   997 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:18.089143  4859 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:18.134873   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45545.1us
I20260709 13:14:18.135592  4879 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:18.143425  4931 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:18.055033   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 61555.4us
I20260709 13:14:18.156208   849 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:18.032605  4875 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.089170  4688 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:18.148691   854 rpcz_store.cc:276] 0709 13:14:17.050638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.050770 (+   132us) service_pool.cc:224] Handling call
0709 13:14:17.051567 (+   797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.051958 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.051988 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.052710 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.801393 (+748683us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:17.801436 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.801504 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.148115 (+346611us) raft_consensus.cc:1685] finished
0709 13:14:18.148145 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.148281 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1147,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":17771,"spinlock_wait_cycles":75648,"thread_start_us":748343,"threads_started":1,"wal-append.queue_time_us":865022}
W20260709 13:14:18.187930   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 200447us
I20260709 13:14:18.051649  4920 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:14:18.167958   851 raft_consensus.cc:3060] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.125006  4793 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.196005  4879 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.199666  1595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37790 (request call id 673) took 1082 ms. Trace:
I20260709 13:14:18.084005  4932 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:18.169780   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 119513us
W20260709 13:14:18.174604  4926 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:14:18.177251  4706 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.164740  4657 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.178355   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2036) took 2521 ms. Trace:
I20260709 13:14:18.179113  4826 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.166249  4754 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:18.215651   928 rpcz_store.cc:276] 0709 13:14:15.656516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.887705 (+231189us) service_pool.cc:224] Handling call
0709 13:14:15.924795 (+ 37090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:16.936839 (+1012044us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:16.937104 (+   265us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.781253 (+844149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:17.781323 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:17.781381 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.178072 (+396691us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400168fd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:18.178125 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.178266 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6148,"prepare.queue_time_us":240,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":7655,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":1898,"spinlock_wait_cycles":2326930816,"thread_start_us":837569,"threads_started":1,"wal-append.queue_time_us":950424}
I20260709 13:14:18.217785  4657 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.172995  4929 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:14:18.219012  4766 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:18.163084   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:18.219429   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2047) took 1197 ms. Trace:
I20260709 13:14:18.220065   921 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:18.219934   929 rpcz_store.cc:276] 0709 13:14:17.021660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.021786 (+   126us) service_pool.cc:224] Handling call
0709 13:14:17.041387 (+ 19601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:17.041807 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.041836 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:17.042394 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:14:17.929231 (+886837us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:17.984664 (+ 55433us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.984893 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.984967 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.219203 (+234236us) raft_consensus.cc:1685] finished
0709 13:14:18.219226 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.219367 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65992,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":47964,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":34776,"thread_start_us":886457,"threads_started":1,"wal-append.queue_time_us":956658}
I20260709 13:14:18.223114  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33108 (request call id 504) took 1121 ms. Trace:
I20260709 13:14:18.223516  1000 rpcz_store.cc:276] 0709 13:14:17.101703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.113612 (+ 11909us) service_pool.cc:224] Handling call
0709 13:14:18.222948 (+1109336us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540022c4d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.223055 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947793152}
I20260709 13:14:18.189146   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:18.089361  4869 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:18.150234  4864 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.192152  4860 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.192291   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1944) took 1120 ms. Trace:
I20260709 13:14:18.193040  4940 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:18.233870   846 rpcz_store.cc:276] 0709 13:14:17.071455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.071592 (+   137us) service_pool.cc:224] Handling call
0709 13:14:18.192033 (+1120441us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54002b8254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.192186 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917233536}
I20260709 13:14:18.194231   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.194470  4875 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:18.196000   848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2260) took 2639 ms (client timeout 1518 ms). Trace:
W20260709 13:14:18.242753   848 rpcz_store.cc:269] 0709 13:14:15.556287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.590060 (+ 33773us) service_pool.cc:224] Handling call
0709 13:14:18.195443 (+2605383us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b540043e754. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.195576 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4660417920}
I20260709 13:14:18.199007  4688 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:18.243743  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:18.239329  4860 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.244263  4688 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.246284  4860 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.228257  4754 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:18.219451  4766 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:18.080637  4815 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.203860  1595 rpcz_store.cc:276] 0709 13:14:17.117352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.117977 (+   625us) service_pool.cc:224] Handling call
0709 13:14:17.568745 (+450768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.571587 (+  2842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.199216 (+627629us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540024ead0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.199515 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035762560}
I20260709 13:14:18.248306  4766 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.249573  4766 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:18.251717  4573 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.219456  4929 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:18.212059   851 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:18.213595   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 579) took 1139 ms. Trace:
I20260709 13:14:18.267493   922 rpcz_store.cc:276] 0709 13:14:17.073974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.144320 (+ 70346us) service_pool.cc:224] Handling call
0709 13:14:18.213378 (+1069058us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400169ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.213528 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1375332096}
I20260709 13:14:18.269129   922 raft_consensus.cc:1275] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:18.227463  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:18.188279  4877 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.228856   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:18.231366  4869 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.232436  4874 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.238729  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:18.241209   969 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.1:43377 timed out after 1.665s (SENT) [suppressed 9 similar messages]
W20260709 13:14:18.275873   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81732.1us
I20260709 13:14:18.277763   967 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.152159  4921 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.242798  4832 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:18.238651  4931 consensus_queue.cc:1059] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:14:18.281811  4945 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:18.209604   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.283952   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.284791   852 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:18.061838  4915 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.207959  4932 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:18.258812   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:18.288231   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.289286   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.265074   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:18.266173  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:18.270394  1596 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:18.232414   819 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.291682   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:18.292433   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:18.292590   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 60272us
I20260709 13:14:18.293077   852 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:18.294148  4865 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:18.294222   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.236860  4826 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.295167  4865 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:18.296118  4874 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:18.253069   742 maintenance_manager.cc:419] P ad0b4d62f81540a8878fe1d1c1552895: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033348
W20260709 13:14:18.280527   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51772.6us
I20260709 13:14:18.267280  4929 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:18.286660   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.269402  4875 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.263777  4780 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.294180   852 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:18.297690   895 leader_election.cc:400] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:14:18.267653  4692 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:14:18.302814   847 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2275) took 1291 ms (client timeout 1665 ms). Trace:
W20260709 13:14:18.314860   847 rpcz_store.cc:269] 0709 13:14:17.011370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.011472 (+   102us) service_pool.cc:224] Handling call
0709 13:14:18.302531 (+1291059us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400409cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.302733 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060763008}
I20260709 13:14:18.274060  4869 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.282300  4945 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:18.315467  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 78915.9us
I20260709 13:14:18.252882  4815 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.282495  4860 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:18.287799  4657 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.310547   895 leader_election.cc:403] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:18.313072   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.320268   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:18.278354  4725 consensus_queue.cc:1048] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:14:18.323698   852 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:18.280752  4877 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.297731  4932 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:18.333000   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:18.312254  4941 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:18.335762   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 49195.7us
I20260709 13:14:18.336848   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.318948  4940 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:18.341845   852 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:18.317126  4945 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:18.324550  4573 raft_consensus.cc:3055] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:18.302517  4783 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.343984   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:18.320991  4767 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.345460  1566 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:18.346362   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:18.353365   998 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:18.354099   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.330178  4657 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9.
I20260709 13:14:18.350371   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:18.351500  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.311450  4826 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.316725  4780 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.345078   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.358088  1003 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:18.315979  4875 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.356829   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:18.342974  4917 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:18.342763  4815 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:18.359676   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.360579   997 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:18.343361  4940 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.376170   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.371692  4783 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.371620   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.372174  4784 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:18.415638  4831 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.416426  4831 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:18.420552   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 60973.1us
I20260709 13:14:18.372534  4451 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:18.348330  4931 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:14:18.374163   998 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:18.370771  4754 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:18.381471  4915 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.382735  4867 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.388767  4921 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.343813  4929 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.354815  4942 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:18.358492  4767 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.346076  4573 raft_consensus.cc:740] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:18.353824  4944 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:14:18.390794  4917 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:18.393201   894 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdfea09a06ff4100a0b80619f35f7008. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:18.462421   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 69392.7us
I20260709 13:14:18.397727  4875 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.404649   853 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:18.406926   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:18.407905   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 53905.4us
I20260709 13:14:18.464680   852 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:18.467100   967 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
I20260709 13:14:18.409818   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.410511  4877 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.410642  4780 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.412987  4826 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.414927  4879 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.407208  4940 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.419365  4831 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.440569  4783 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.420008  4872 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:18.433069  4941 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:18.468400   967 leader_election.cc:403] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:18.469852   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.472620   847 raft_consensus.cc:2410] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:18.464345   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.440032  4942 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.457527  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:18.480733  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 127529us
I20260709 13:14:18.478709  4720 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:18.479538  4784 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:18.438163  4451 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.504771   852 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:18.529115   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.533250   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.503252   922 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:18.543125   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:18.482072  4335 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:18.477491  4955 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:18.482097  4766 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:18.549528  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48028.7us
I20260709 13:14:18.549793   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.500283  4915 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:18.500388  4826 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.544013   922 raft_consensus.cc:2376] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12. Re-sending same reply.
I20260709 13:14:18.506728  4783 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.551525  4335 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.504483  4784 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:18.577744   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 94232us
I20260709 13:14:18.505105  4942 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:18.580142   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.504706  4831 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
I20260709 13:14:18.581038   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.583578   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:18.576056  4936 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:18.605823   922 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:18.607069   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.608533   847 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:18.610268   922 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:18.505651  4720 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:18.648921   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45169.7us
I20260709 13:14:18.650395   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.652822  4451 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.580776  4547 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:18.505928  4879 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.544983  4780 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:18.585042  4924 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.507128  4573 consensus_queue.cc:260] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.608032  4936 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:18.715430  1599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45838 (request call id 415) took 1570 ms (client timeout 1666 ms). Trace:
W20260709 13:14:18.768720  1599 rpcz_store.cc:269] 0709 13:14:17.145281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.149656 (+  4375us) service_pool.cc:224] Handling call
0709 13:14:18.715210 (+1565554us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b5400270954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.715356 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327950080}
I20260709 13:14:18.608511  4932 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.717406   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.608088  4874 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:18.587983  4766 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.584281  4915 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:18.710809  4859 consensus_meta.cc:287] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.007s
I20260709 13:14:18.591229  4826 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.717236   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:18.600268  4917 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.727872  4386 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:18.779968  4951 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
W20260709 13:14:18.780665  4954 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:14:18.785163  4720 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.781652  4784 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.797955   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.798220  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.798691   997 raft_consensus.cc:2368] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:14:18.712963  4955 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:18.801437  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:18.801821   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:18.803177   996 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:18.785471  4952 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:14:18.759239  4547 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:18.788714  4754 consensus_queue.cc:1175] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 11 status { last_received { term: 11 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 11 index: 5 } }
I20260709 13:14:18.753895  4451 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:18.791036   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 73895.5us
I20260709 13:14:18.793745  4959 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:18.789937  4932 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.796581   998 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:14:18.775159  4924 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.792052  4859 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:18.794250  4573 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.778808   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:18.823813  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 97756.3us
I20260709 13:14:18.795364  4706 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:18.798177  4784 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:18.801458  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 215802us
I20260709 13:14:18.817709   821 leader_election.cc:400] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:14:18.820403  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.787124  4874 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.803026  4783 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.817602   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:18.806566  4867 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.830232  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.831724   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.829630   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:18.833663   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:18.916492   922 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:18.917484   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:18.836247  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:18.918820  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82688.1us
I20260709 13:14:18.919786  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.922339  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:18.835177   821 leader_election.cc:403] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:18.796613  4386 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:18.843101   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.843780   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:18.929430   998 raft_consensus.cc:2393] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:18.850608   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:18.858703   996 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:18.917969  1004 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:18.923024  1596 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:18.923069   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 105517us
W20260709 13:14:18.924283   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 94738.1us
I20260709 13:14:18.933377   896 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:18.926949  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:18.800860  4936 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.950263  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:18.955694  1599 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:18.956735   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 106227us
I20260709 13:14:18.957790   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:18.821947  4451 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.914690  4780 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:18.935977  4720 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:18.929064  4335 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:18.975219   821 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:18.977178  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:18.928495  4783 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:14:18.995920   968 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:18.982827  4720 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:18.936537  4961 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:18.983323  1601 raft_consensus.cc:2393] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:18.991731   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.824442  4959 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.971903  4932 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:18.964938  4879 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:18.971002  4941 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:19.006057  4924 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:18.974599  4872 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:19.012270   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:18.998507  4936 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:19.018424   922 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.000906  4720 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:18.977460  4875 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:14:19.004886  4706 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:19.035037  4451 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.038199  4874 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.039017  4547 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.014160  4872 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:19.040203  4872 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:19.042416  4965 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:19.043104   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 1954) took 1056 ms. Trace:
I20260709 13:14:19.047490   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.048393  4386 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.011893  4932 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:19.032774  4915 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.049482  4932 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.050582  4780 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:19.031378  4869 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
I20260709 13:14:19.053592  4869 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:18.955233  4955 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.018914  4879 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:19.056499  4879 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:19.048671   850 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2287) took 1876 ms (client timeout 1921 ms). Trace:
W20260709 13:14:19.065465   850 rpcz_store.cc:269] 0709 13:14:17.172078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.218310 (+ 46232us) service_pool.cc:224] Handling call
0709 13:14:19.048490 (+1830180us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000ca8d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.048618 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596621056}
I20260709 13:14:19.068315   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:19.069141   995 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:19.071466   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.073593   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:19.074308   996 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:19.076715  4879 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.077299   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:19.021463  4969 raft_consensus.cc:2804] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:19.080684  4617 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.055588  4869 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:19.082072   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 585) took 1075 ms. Trace:
I20260709 13:14:19.082442   926 rpcz_store.cc:276] 0709 13:14:18.007280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.012618 (+  5338us) service_pool.cc:224] Handling call
0709 13:14:18.024886 (+ 12268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.137429 (+112543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.137466 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.137517 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:18.137558 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.081845 (+944287us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540017a554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:19.081891 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.082015 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258340736}
I20260709 13:14:19.084190  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.085363  1004 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:19.021692  4924 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:19.043478   855 rpcz_store.cc:276] 0709 13:14:17.986301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.131437 (+145136us) service_pool.cc:224] Handling call
0709 13:14:18.304639 (+173202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.304846 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:18.309177 (+  4331us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.041672 (+732495us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18001cf6c0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0d026e74bc 00007f0cffa3548b 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df23a
0709 13:14:19.041933 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:19.041987 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:19.042029 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.042092 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.042937 (+   845us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400409cd0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.043051 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":1675,"raft.queue_time_us":4635,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":404368896,"thread_start_us":13252,"threads_started":1,"wal-append.queue_time_us":68104}
W20260709 13:14:19.096828  4940 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:14:19.097029   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:19.097324  4940 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.098219   895 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:19.099262   895 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:19.029588  4826 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.104811  1002 raft_consensus.cc:2410] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:14:19.099903  1001 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:19.100301  4915 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:19.100853   998 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:14:19.091002   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 157705us
I20260709 13:14:19.116696   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.116813   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:19.100473  4706 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:19.099793  4451 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:19.115257   922 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:19.119823   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:19.120481   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:19.118377  4945 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.003s
I20260709 13:14:19.122283  4617 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:19.125100  4720 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.126576   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:19.129673   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 584) took 1144 ms. Trace:
I20260709 13:14:19.081835  4874 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.099419  4940 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:19.100330  4969 raft_consensus.cc:697] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:19.152076   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.148589   930 rpcz_store.cc:276] 0709 13:14:17.985027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.985152 (+   125us) service_pool.cc:224] Handling call
0709 13:14:17.985415 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.161918 (+176503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.161946 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.161986 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:18.162016 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.129441 (+967425us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400328cd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:19.129487 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.129618 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405457664}
I20260709 13:14:19.154062  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.125748  4960 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:19.125337  4959 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.170866  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.170992   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:19.163357   995 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
W20260709 13:14:19.172075   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.156489  1003 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:19.166348   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.173301   926 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:19.166590   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 696600us
I20260709 13:14:19.136555  4915 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:19.179184  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:19.180171   998 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:19.180765  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:19.169755   996 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:19.176381   926 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:19.185861   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.186640   997 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:19.152336  4976 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:19.194548   855 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:19.167318  4960 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:19.196322  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:19.216545   821 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:19.147919  4945 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.219677  4960 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.220050  1003 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:19.220525   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:19.224216   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.189469  4865 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:19.185420  4826 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:19.225317   930 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:19.153368  4874 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.241791   925 raft_consensus.cc:2410] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:19.230623   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:19.167454  4942 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.234570  4969 consensus_queue.cc:237] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.235524  4968 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:19.238483   929 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:19.241430   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:19.252957   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:19.252418  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 73586.3us
I20260709 13:14:19.220592  4780 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:14:19.246203  4875 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:19.250092  4867 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:19.234618  4973 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:14:19.244227  4826 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:19.299641  4864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.007s	sys 0.009s
W20260709 13:14:19.317351   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 64491.6us
I20260709 13:14:19.330204   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:19.302615  4966 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.241919  4865 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.334893  4720 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.248172  4968 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.338598  4968 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.340749  4968 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:19.341202  4968 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:19.342226  4968 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.302251  4867 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:19.333566  4864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.008s	sys 0.009s
I20260709 13:14:19.301162  4981 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:14:19.353076  4867 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:19.372323  4945 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:19.372746  4978 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:14:19.374401  4987 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:14:19.382452  4865 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.384615  4997 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.361748  4780 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:19.380692  4875 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:19.377590  4924 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:19.376991  4874 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.401605   818 leader_election.cc:400] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8.
I20260709 13:14:19.354108  4966 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.405757   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:19.392292  4780 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:19.397234  4826 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.402575  4865 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:14:19.434494  4946 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 1.047s	user 0.007s	sys 0.003s
I20260709 13:14:19.430266   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:19.437269  4986 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:14:19.438292  4988 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:19.440076  4879 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:19.425163  4865 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:19.441603  4865 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.443114  4865 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:19.448175  4985 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:19.421422  4780 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:19.385270  4997 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.470100  4970 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.461969  4942 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.471825   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 590) took 1206 ms. Trace:
I20260709 13:14:19.472857  4997 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.472278   928 rpcz_store.cc:276] 0709 13:14:18.265678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.266350 (+   672us) service_pool.cc:224] Handling call
0709 13:14:18.269734 (+  3384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.311781 (+ 42047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.311814 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.311856 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:18.311885 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.471531 (+1159646us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b5400082354. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:19.471588 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.471770 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96256256}
I20260709 13:14:19.476181   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:19.475684   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 156936us
I20260709 13:14:19.485885   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.485742   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.490921  4970 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.491763  4970 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.494477  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.495357  1004 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:19.496446   928 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:19.477255  4860 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.502941   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:19.505602   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.508332   998 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:19.491014  4494 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:19.500118  4868 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:19.510731  4494 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:19.500231  4997 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:19.518496  4872 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.518065  4966 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.521303   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1951) took 1536 ms. Trace:
I20260709 13:14:19.522436   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46364 (request call id 1949) took 1585 ms. Trace:
I20260709 13:14:19.533973   851 rpcz_store.cc:276] 0709 13:14:17.985049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.079989 (+ 94940us) service_pool.cc:224] Handling call
0709 13:14:19.521044 (+1441055us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b540023ad54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.521230 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199615744}
I20260709 13:14:19.522658   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:19.468199  4875 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.528976  4997 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.503850  5005 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.534140   849 rpcz_store.cc:276] 0709 13:14:17.937293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.978580 (+ 41287us) service_pool.cc:224] Handling call
0709 13:14:19.522295 (+1543715us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5400177ad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.522384 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38803200}
I20260709 13:14:19.519289  4868 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.524539  4960 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.510509  5004 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.509104  4860 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.542706  4968 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:19.543337   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:19.562182   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:19.567173  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:19.557459  4968 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.560076  4879 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:19.574306  5000 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.547683  4494 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.567991  4657 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:19.575098  4494 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:19.545193  5004 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.562609  4970 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.571264  4879 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:19.577147  4867 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.578485  4997 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:19.578547   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 337129us
I20260709 13:14:19.576469  4657 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:19.544453  5005 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.571441  4968 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:19.586467   926 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:19.585553  4879 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.601203   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2067) took 1061 ms. Trace:
I20260709 13:14:19.601605   926 rpcz_store.cc:276] 0709 13:14:18.540099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.585689 (+1045590us) service_pool.cc:224] Handling call
0709 13:14:19.586153 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.600875 (+ 14722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.601140 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:19.589859   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2059) took 1058 ms. Trace:
I20260709 13:14:19.588280  5004 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.574867  5000 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.579012  4860 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:19.599226  4868 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:19.606963   930 rpcz_store.cc:276] 0709 13:14:18.531548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.170705 (+639157us) service_pool.cc:224] Handling call
0709 13:14:19.589701 (+418996us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b5400169ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.589810 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122301312}
I20260709 13:14:19.618769   930 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:19.610196   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:19.620534  4767 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.606345  4725 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:14:19.622519   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 587) took 1412 ms. Trace:
W20260709 13:14:19.623529  4947 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.235s	user 0.004s	sys 0.002s
I20260709 13:14:19.625245   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 589) took 1406 ms. Trace:
I20260709 13:14:19.625571  4657 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.625662   921 rpcz_store.cc:276] 0709 13:14:18.218663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.220861 (+  2198us) service_pool.cc:224] Handling call
0709 13:14:18.221168 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.221483 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.221510 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.221981 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.307960 (+ 85979us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:18.308027 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.308129 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.587572 (+1279443us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:19.624926 (+ 37354us) raft_consensus.cc:1685] finished
0709 13:14:19.624951 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.625178 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1532,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"spinlock_wait_cycles":58403072,"thread_start_us":85543,"threads_started":1,"wal-append.queue_time_us":111091}
I20260709 13:14:19.623011  4960 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.633067   930 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:14:19.633461   930 pending_rounds.cc:85] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 9
I20260709 13:14:19.633738   930 pending_rounds.cc:107] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:14:19.633970   930 raft_consensus.cc:2889] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:19.593283  5007 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:19.630520  5000 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.638037  5008 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:19.638586  5009 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:14:19.639374  4867 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:19.640807  4780 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.623000   927 rpcz_store.cc:276] 0709 13:14:18.210230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.210387 (+   157us) service_pool.cc:224] Handling call
0709 13:14:18.210689 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.315591 (+104902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.315637 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.315676 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:18.315712 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.622177 (+1306465us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54001b0654. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:19.622263 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.622445 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240841472}
I20260709 13:14:19.625590  4766 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:19.672145   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.659246   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.678133  5008 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:19.716634  4780 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:19.723085  5011 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:14:19.723425  5008 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.736354   927 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:19.714774  4879 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:19.740027   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.741298   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:19.741649  4879 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.742476   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.730178  5005 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:19.752578   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.753481   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.754711   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:19.762902   930 raft_consensus.cc:2410] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:19.727301  4657 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:19.764045   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:20.088316   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.762650  4881 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:20.127282  4929 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.047173  1596 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:20.129918  4831 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:20.130107   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:20.130465  4831 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.130884   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.131765  4860 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:19.874102   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2057) took 1343 ms. Trace:
I20260709 13:14:20.043848  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:19.931967   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2066) took 1392 ms. Trace:
I20260709 13:14:20.124826  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:19.872604   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2071) took 1055 ms. Trace:
I20260709 13:14:20.051182  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:20.128029   927 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 4.
I20260709 13:14:20.134268   928 rpcz_store.cc:276] 0709 13:14:18.539805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.497546 (+957741us) service_pool.cc:224] Handling call
0709 13:14:19.500939 (+  3393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.501248 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:19.721307 (+220059us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.931584 (+210277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:19.931660 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:19.931710 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.931779 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.931905 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1233,"prepare.queue_time_us":220275,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":49,"raft.queue_time_us":1371,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":425677,"threads_started":2}
I20260709 13:14:19.873420   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 592) took 1500 ms. Trace:
I20260709 13:14:20.134148   922 rpcz_store.cc:276] 0709 13:14:18.816686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.917223 (+100537us) service_pool.cc:224] Handling call
0709 13:14:19.872388 (+955165us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54004466d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.872529 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230560000}
I20260709 13:14:20.128190  4881 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:19.961440  4872 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.136548   924 rpcz_store.cc:276] 0709 13:14:18.372980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.373149 (+   169us) service_pool.cc:224] Handling call
0709 13:14:18.373430 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.416347 (+ 42917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.416380 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.416422 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:18.416458 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.873158 (+1456700us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54002314d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:19.873215 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.873358 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98248576}
I20260709 13:14:20.132921   925 rpcz_store.cc:276] 0709 13:14:18.530843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.545574 (+ 14731us) service_pool.cc:224] Handling call
0709 13:14:19.873871 (+1328297us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400175f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.874039 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464135552}
I20260709 13:14:20.164788  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:20.134593  1601 raft_consensus.cc:2376] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12. Re-sending same reply.
I20260709 13:14:19.726326  4766 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:20.164083  4932 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:20.100109  5020 log.cc:927] Time spent T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.384s	user 0.002s	sys 0.003s
W20260709 13:14:20.091187  5014 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:14:19.936273   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2070) took 1326 ms. Trace:
I20260709 13:14:19.730434  4921 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:20.215260   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2075) took 1237 ms. Trace:
I20260709 13:14:19.765576   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.183820  4692 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.132261  4860 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.090752  4964 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:20.091017  4688 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:20.208266   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.212543   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79950.3us
I20260709 13:14:20.060352  4700 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.229614  1000 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.181936  4963 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.218741   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.238471   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.219445   928 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:20.050769  4793 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.050225  4830 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.240178   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.240468  4830 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.241158   928 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:14:20.222503   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 130092us
I20260709 13:14:20.211633  4832 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.226696   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.171622  4993 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:20.232025  1595 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:20.231900  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.233783  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:20.234359   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.235351   995 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.248579   996 raft_consensus.cc:2368] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:20.249060   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.250362  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.235906  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:20.251353  1598 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:20.213291   929 rpcz_store.cc:276] 0709 13:14:18.610110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.047218 (+437108us) service_pool.cc:224] Handling call
0709 13:14:19.936069 (+888851us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400444dd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:19.936207 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437596928}
I20260709 13:14:20.101658  4968 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.223317   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.127406  4657 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.265787  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.214689  4940 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:20.245918   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.253005   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.259089  1601 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:20.266812   928 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:20.216099  4932 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.217875  4872 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:20.270925   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.273586   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.274485   850 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:20.275030   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.291581   855 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:20.292349   855 pending_rounds.cc:85] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 3
I20260709 13:14:20.208542  4766 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.292620   855 pending_rounds.cc:107] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:20.292716   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:20.314983   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 102815us
I20260709 13:14:20.317454   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:20.291409   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.291683   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:20.221485  4692 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.316257   855 raft_consensus.cc:2889] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.332965   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.333648   855 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.333899  4692 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.334611  1001 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:20.291188   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.236676  4706 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.229652  4831 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.318295   854 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:20.342794   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.343614   854 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.343564  4997 consensus_queue.cc:1059] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:20.345546  4968 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.347973  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.350296   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.352008   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.352557   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.351367  4706 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.352782   846 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 6.
I20260709 13:14:20.353307   997 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 7: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:20.244009  4700 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.321234   853 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:20.223096  4921 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.321046  4872 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.246381  4832 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.291118  4793 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.357366  4832 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.334298   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.359450   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.361505   852 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:20.254570  4859 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:20.362272  4859 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.246780  4881 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.364396  4859 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.292173  4783 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.320103   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 593) took 1931 ms. Trace:
I20260709 13:14:20.365363  4783 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.365751   923 rpcz_store.cc:276] 0709 13:14:18.389185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.389354 (+   169us) service_pool.cc:224] Handling call
0709 13:14:18.389708 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.530996 (+1141288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:19.531028 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.536272 (+  5244us) log.cc:844] Serialized 26 byte log entry
0709 13:14:19.635110 (+ 98838us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:19.635149 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.082495 (+447346us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003caad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:20.082540 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.090406 (+  7866us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:20.319892 (+229486us) raft_consensus.cc:1685] finished
0709 13:14:20.319915 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.320048 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5926,"raft.queue_time_us":6687,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4149,"spinlock_wait_cycles":2648988288,"thread_start_us":95059,"threads_started":1,"wal-append.queue_time_us":274286}
I20260709 13:14:20.246008  4963 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.237883  4780 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:14:20.367559  4783 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.369295  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.370913  1002 raft_consensus.cc:2368] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:20.324609  4932 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.354195   922 raft_consensus.cc:2410] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:20.354079   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.357973  4872 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.293493  4867 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:20.357234  4860 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.374423   855 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:20.357929  4793 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.243113  4688 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.377435   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.388744   850 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.365515   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.389228   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.238122  4964 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.376545   854 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:20.380086   846 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:20.394131   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.394933   852 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: [9.5-9.6]
I20260709 13:14:20.396953   854 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:20.399255   855 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:20.369688  4963 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.404057   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.401185   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.405933   923 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:20.384588  4968 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:20.416934  4964 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.417464   850 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:20.422395   848 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.392189  4780 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:20.428751   853 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:20.267891  4993 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.436832   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.438110   923 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:20.440877   998 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:20.444411  4688 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.446313  4793 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.381976  4867 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.456101  4700 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.458393   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.459069   922 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:20.462265   851 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:20.434180  4968 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:20.470299   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.471892   922 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
W20260709 13:14:20.481716   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 350872us
I20260709 13:14:20.215764   921 rpcz_store.cc:276] 0709 13:14:18.978122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.627491 (+649369us) service_pool.cc:224] Handling call
0709 13:14:19.631589 (+  4098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.631843 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:19.724696 (+ 92853us) log.cc:844] Serialized 14 byte log entry
0709 13:14:20.214760 (+490064us) log.cc:844] Serialized 14 byte log entry
0709 13:14:20.214925 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:20.214969 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:20.215011 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.215076 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.215203 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":90318,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":3260,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9037,"spinlock_wait_cycles":9538304,"thread_start_us":581679,"threads_started":2}
I20260709 13:14:20.483327  4830 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.490011   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.499464  4860 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:20.502552   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.503274   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.506418   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.486900  4868 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:14:20.490401  4929 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:14:20.506531   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:20.488899  4928 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:14:20.509155   922 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:20.501943  4964 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:20.506418  4859 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:20.489915  4968 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.550089   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:20.563634   927 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:20.564240   927 pending_rounds.cc:85] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:14:20.564921   927 pending_rounds.cc:107] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:20.565835   927 raft_consensus.cc:2889] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.567101   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.567736   923 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:20.505314  4860 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:14:20.568840  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.569270   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.569811   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:20.530236  4993 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.570680   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:20.568501   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.569412  1000 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.574568   996 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:20.572504   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.577736   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.580332   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.581923   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.582846   995 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:20.579257   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.586862   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.587725   996 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:20.596184  4793 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.596812  4793 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.599638  4793 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.602310  5000 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:20.602794  5000 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:20.603652  5000 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.605612  5000 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.564225  4867 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.565713  4964 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:20.613687   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.615159   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.568475  4706 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.615217   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.618052   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.620054   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:20.627918  1000 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:20.592612  4700 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.632597   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.634251   848 raft_consensus.cc:2393] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:20.635490   923 raft_consensus.cc:1240] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:20.638347   997 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:20.640305   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.640905   996 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:20.632409  4706 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:14:20.630004  4964 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:20.655443   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.640679  4793 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:20.671552   997 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:20.637413  4700 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.660859  4793 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:20.684149  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 320987us
W20260709 13:14:20.686312   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 459290us
I20260709 13:14:20.688488  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.689397  1000 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:20.691407  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:20.694245   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 376807us
I20260709 13:14:20.695129  1000 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:20.697911  1000 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.700215   927 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:20.701929  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:20.702183  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.702741   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.703210   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:20.703397   930 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:20.703831   923 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:20.705924   997 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.684826  4793 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.711560   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:20.716411  4881 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.721267   852 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:20.725375  4688 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:20.725840  4688 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:20.726667  4688 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.730464  1004 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:20.731956   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.732470  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.732962   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.731091  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:20.733193   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.731361  4688 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.733999   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.734443   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:20.735073   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.735191   995 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:20.737043   854 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:20.737454   854 pending_rounds.cc:85] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
I20260709 13:14:20.737708   854 pending_rounds.cc:107] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:20.737962   854 raft_consensus.cc:2889] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.738152   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.739094   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.739693  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:20.739959   855 raft_consensus.cc:2410] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:20.740468  4836 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.740900  4836 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.741295  4836 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.742725  4955 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:20.744083  4955 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:20.744529  4955 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:20.745488  4955 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.747000  4955 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.748361   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.749327  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:20.753894   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:20.768605  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45966.1us
I20260709 13:14:20.770612   969 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:20.771060   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 59563us
I20260709 13:14:20.772002   818 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.767956  4763 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:20.802745   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.806988   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.774217   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.806370  4875 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:14:20.810227   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.807910   848 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
W20260709 13:14:20.833981   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 59833.1us
I20260709 13:14:20.712522  5001 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:20.772835  4688 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.773003  4780 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:20.835012   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.809850  5024 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:20.841248   969 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.862357   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.776645  4993 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:20.697512  4964 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.832885  4867 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:20.876418  4921 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.876606  4867 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:20.876891  4879 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:20.877272  4879 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:20.877334  4867 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:20.850342  4700 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.830978  4872 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:14:20.878118  4879 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.842428  4928 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:20.882193  4928 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:20.882766  4928 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.773861  4832 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.842592  4920 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:14:20.842821  5001 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.885546  4832 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:20.886739  4836 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.868580  4780 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:20.889362  4836 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.890028  4836 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:20.890450  4836 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:20.890228  4780 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.892204   999 raft_consensus.cc:2368] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:20.857586  4793 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.868886   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:20.871927  4932 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:20.872390  4860 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:20.875478  5005 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:14:20.907258   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:20.887130  5001 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:20.863692  4688 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.772925  5000 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:14:20.905550   851 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:20.879362  4879 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.920456  4766 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:20.921119  4766 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:20.926321  4924 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:20.926854  4924 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:20.927240  4924 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.891355  4836 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:20.906970  4875 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:20.930424  4875 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.909195  5024 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:20.928913  4793 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:20.917268   968 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.001s	sys 0.026s
I20260709 13:14:20.938004   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:20.945443  4494 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:20.972419  4494 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:20.973330  4494 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:20.945779  4868 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:20.976763  4868 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:20.977123  4657 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:20.979239  5017 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.825s	user 0.008s	sys 0.000s
I20260709 13:14:20.979846   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:20.982836  4868 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:20.987772   968 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.001s	sys 0.028s
I20260709 13:14:20.660921 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:20.971246   855 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:21.002192  4657 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:21.002805  4657 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.005124  4657 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:21.008423   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 1.07829e+06us
W20260709 13:14:21.016717   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 384761us
I20260709 13:14:21.018306   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:21.022045  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:21.023188  1002 raft_consensus.cc:2410] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:20.987323   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 120) took 9280 ms. Trace:
W20260709 13:14:21.051455   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.052080   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.052553   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.917613  4860 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:20.911159  4832 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:14:21.089288   750 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 6 to 9, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 9 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
W20260709 13:14:21.085129   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51610.9us
I20260709 13:14:21.093425  4688 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:21.095170  4875 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:21.019742  5024 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:21.028079   751 rpcz_store.cc:276] 0709 13:14:11.770276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.770549 (+   273us) service_pool.cc:224] Handling call
0709 13:14:20.987250 (+9216701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:16.205420 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:16.290559 (+ 85139us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:16.293142 (+  2583us) write_op.cc:435] Acquired schema lock
0709 13:14:16.295829 (+  2687us) tablet.cc:662] Decoding operations
0709 13:14:16.343593 (+ 47764us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:16.343697 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:16.343738 (+    41us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:14:16.344455 (+   717us) tablet.cc:701] Row locks acquired
0709 13:14:16.344491 (+    36us) write_op.cc:260] PREPARE: finished
0709 13:14:16.344693 (+   202us) write_op.cc:270] Start()
0709 13:14:16.344804 (+   111us) write_op.cc:276] Timestamp: P: 1783602856344628 usec, L: 0
0709 13:14:16.344826 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:16.345684 (+   858us) log.cc:844] Serialized 2434 byte log entry
0709 13:14:17.580007 (+1234323us) op_driver.cc:464] REPLICATION: finished
0709 13:14:18.084036 (+504029us) write_op.cc:301] APPLY: starting
0709 13:14:18.089834 (+  5798us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:18.157278 (+ 67444us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:18.170826 (+ 13548us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:18.216650 (+ 45824us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:18.216865 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:14:18.216914 (+    49us) write_op.cc:312] APPLY: finished
0709 13:14:18.217139 (+   225us) log.cc:844] Serialized 53 byte log entry
0709 13:14:18.289513 (+ 72374us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:18.289628 (+   115us) write_op.cc:454] Released schema lock
0709 13:14:18.289670 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:18.290059 (+   389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:18.290088 (+    29us) write_op.cc:454] Released schema lock
0709 13:14:19.433871 (+1143783us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
Metrics: {"mutex_wait_us":1181187,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":449575,"apply.run_cpu_time_us":14355,"apply.run_wall_time_us":238169,"num_ops":6,"prepare.queue_time_us":146626,"prepare.run_cpu_time_us":97481,"prepare.run_wall_time_us":348253,"raft.queue_time_us":745248,"raft.run_cpu_time_us":13619,"raft.run_wall_time_us":452194,"replication_time_us":1160944,"spinlock_wait_cycles":51927424,"thread_start_us":1039520,"threads_started":5,"wal-append.queue_time_us":440337,"wal-append.run_cpu_time_us":143165,"wal-append.run_wall_time_us":3547002}]]}
I20260709 13:14:21.099988  4940 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.100826  4940 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.102197  4924 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.091948  5025 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.943733  4863 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.141s
I20260709 13:14:21.062304  4793 raft_consensus.cc:2764] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:14:21.103543  5025 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.096452   750 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 6 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.099277  4872 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.110285   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:21.125294   851 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:21.087675  4832 consensus_queue.cc:1243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:21.048063  4657 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:14:21.122072   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 5 to 8, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 8 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.128875   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:21.132520   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:21.137833   750 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 3 to 5, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 5 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:21.108777   855 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:21.157673  4932 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:21.158301  4932 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.158756  4932 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:21.162949  4793 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:21.165148  5025 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:21.139468   847 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:21.132424  4940 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:21.184706   750 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 9 to 10, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 10 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.133895  4872 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:21.210093  4706 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.013s	sys 0.029s
W20260709 13:14:21.226310  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 456743us
W20260709 13:14:21.226428  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 322669us
W20260709 13:14:21.229390  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 422557us
W20260709 13:14:21.225354  4706 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.014s	sys 0.029s
W20260709 13:14:21.238427   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 403455us
W20260709 13:14:21.228315   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 499272us
I20260709 13:14:21.235682  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:21.171217  4860 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:21.228693  4783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.011s
W20260709 13:14:21.261257  4783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.010s	sys 0.012s
I20260709 13:14:21.262058  5036 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:21.263201  5036 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:21.239435   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.257187  4830 consensus_queue.cc:1059] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:21.210464  5024 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.280071   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:21.274667  5032 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:14:21.272686  4955 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:21.304502   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.304903   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:21.306596  5025 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:21.311765  5025 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.314154  5025 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:21.306576   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.282681  4830 raft_consensus.cc:3055] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:21.315234  4830 raft_consensus.cc:740] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:21.305612  5044 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:21.263577  5036 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:21.273892  4872 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:21.327889   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:21.318112   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 231035us
W20260709 13:14:21.319054   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 509958us
I20260709 13:14:21.334478   995 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:21.322885   896 leader_election.cc:400] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:21.335632   896 leader_election.cc:403] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:21.211647   750 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 5, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.327270   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2308) took 1053 ms. Trace:
I20260709 13:14:21.337033   849 rpcz_store.cc:276] 0709 13:14:20.273517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.274965 (+  1448us) service_pool.cc:224] Handling call
0709 13:14:20.275362 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:20.319853 (+ 44491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:20.319900 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:20.457454 (+137554us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b2c00000c88. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:14:20.461364 (+  3910us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.700012 (+238648us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:20.700087 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.700186 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:21.316943 (+616757us) raft_consensus.cc:1685] finished
0709 13:14:21.316972 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:21.327166 (+ 10194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54260,"raft.queue_time_us":101339,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6540,"spinlock_wait_cycles":66509056,"thread_start_us":228763,"threads_started":1,"wal-append.queue_time_us":639833}
I20260709 13:14:21.338438   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.282954  4924 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:21.344074  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 116361us
I20260709 13:14:21.304020  4868 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.279744  4993 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:21.340927   750 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:21.373538   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 130181us
I20260709 13:14:21.378691   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:21.305780  4955 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.379854   750 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.343657  4832 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.271287  4928 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.318588  4836 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.382678  4955 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:21.386895  4706 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.387907  5040 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:21.387946  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:21.357259  4879 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.392717   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:21.393438   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:21.393656   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.393880   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:21.395252   999 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:21.419593   896 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.379462   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:21.329859  4964 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:21.395376   750 catalog_manager.cc:5697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 3 to 6, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 6 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.390447  4932 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:21.415223  4494 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.409920  4783 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:14:21.422796   896 leader_election.cc:400] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:21.424835   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.395638  4875 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:21.343336  5044 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:21.319718  4688 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
W20260709 13:14:21.425530   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46318.5us
I20260709 13:14:21.423432  4964 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:21.316145  4830 consensus_queue.cc:260] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:21.434825   750 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 3 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:21.359011  4872 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:14:21.442900   896 leader_election.cc:403] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:14:21.481389   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 58797.8us
I20260709 13:14:21.446632   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:21.451305   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 322525us
I20260709 13:14:21.407135  4767 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:14:21.402645  4868 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:21.427541   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.430557  5045 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:21.429397  4766 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.474417   895 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:21.487778  5045 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:21.491746   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:21.492596   855 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.493723   851 pending_rounds.cc:85] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:14:21.494050   851 pending_rounds.cc:107] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:21.494328   851 raft_consensus.cc:2889] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:21.495680   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:21.444828  4657 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:21.498486   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51949us
I20260709 13:14:21.499233   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:21.500505   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.501720   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:21.470120  4929 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:21.496385   848 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:21.506817   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:21.485661   969 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:21.509642   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:21.444844  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 1997) took 1213 ms. Trace:
I20260709 13:14:21.479521  4964 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.470911  5044 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:21.472497  4997 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:21.514719  1003 rpcz_store.cc:276] 0709 13:14:20.231360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.232420 (+  1060us) service_pool.cc:224] Handling call
0709 13:14:20.237316 (+  4896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.237512 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:20.686493 (+448981us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.444154 (+757661us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.444463 (+   309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:21.444512 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:21.444558 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.444637 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:21.444771 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":856536,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":83,"raft.queue_time_us":20678,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":26961,"thread_start_us":1162489,"threads_started":3,"wal-append.queue_time_us":581660}
W20260709 13:14:21.512881  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 275467us
I20260709 13:14:21.488265  4766 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.522505   999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:21.528766  1001 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:21.531574  4921 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:21.479871  5054 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:21.424284  5036 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:21.514672  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 279094us
I20260709 13:14:21.520344   855 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:21.488332  5045 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:21.476909  4763 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:14:21.472148  4993 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.676482   854 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:21.604735   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:21.596969   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 67205.5us
I20260709 13:14:21.647852   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:21.596773   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 90221us
I20260709 13:14:21.662973  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:21.879719   999 raft_consensus.cc:606] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.879978  1592 raft_consensus.cc:2393] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:21.806470  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 400689us
I20260709 13:14:21.625137  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:21.879914  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:21.637032   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 153282us
I20260709 13:14:21.637737  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:21.620504  4657 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:21.737632  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:21.886641  4657 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:21.887427  4657 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.597746  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:21.596909   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 60317.9us
I20260709 13:14:21.810560  4706 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:21.894572   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:21.671594   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.610541  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:21.897423   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.880293   999 raft_consensus.cc:614] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:21.900657   968 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:21.905049  1000 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:21.885547   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.920130  1000 raft_consensus.cc:606] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:21.747123  5069 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:21.953364  1000 raft_consensus.cc:614] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:21.953469  5057 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:21.955814  4921 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:21.957150   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:21.957971  5036 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.957594  4921 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.521786  5055 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:21.956039  5044 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:21.959352   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 55928.6us
I20260709 13:14:21.905803   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:21.983752  5070 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.984525  5070 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.985265  5065 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:21.985747  5065 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.986124  5065 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.534778   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:21.995973  4881 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:21.996798   847 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:21.997133  4706 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:21.998004  4706 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.000154  5071 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:22.001237  4921 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.001130  5071 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:21.610595  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:21.754871  4928 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:21.595788  4780 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:21.749362  4955 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:22.008275  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 128835us
I20260709 13:14:22.008960  4964 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.009541  4657 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.591071  5046 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:21.746268  5060 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:14:21.513188  5039 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:14:21.764233  5052 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:21.982738  5073 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:21.757923  4830 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:22.013386   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 119389us
I20260709 13:14:22.009625  4955 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:21.521019  4879 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:22.018939   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.747349  4868 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.019862  4688 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.020008  5055 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:22.021442  4832 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:22.020671  4700 consensus_queue.cc:1175] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64f2cb5b0e174002811ef197f9eec829" responder_term: 10 status { last_received { term: 9 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:21.749900  5035 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:22.046530  5035 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:21.670805  4929 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:22.046517  5065 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.048023  4929 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.050760  5065 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.052982  4766 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10.
I20260709 13:14:22.009476  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:22.057945  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 48659.4us
I20260709 13:14:22.059741  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:21.648090  5049 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:14:22.016724  5054 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:21.625592  5040 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.035945  1593 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:14:22.040589  4706 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:21.732882  4932 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:22.068751  4494 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.626s	user 0.006s	sys 0.008s
I20260709 13:14:21.520783  4940 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:22.069393  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.069602  5005 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:22.070451  5005 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.072914  5005 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.075747  5068 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:22.076610  5068 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.063814  4881 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.078594  4881 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.079246  5068 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:21.696934  5053 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:22.054437  5081 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:22.054443  5035 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.514458  4875 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.024715  5070 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.068214  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.090655  4879 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:22.069666  4940 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.091732  4940 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.619124  5000 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.481s
W20260709 13:14:22.094560   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 189258us
W20260709 13:14:22.045140   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 159967us
I20260709 13:14:22.095634   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:22.096758  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.081102  4881 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.035259  4832 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.084399   852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2329) took 1148 ms. Trace:
I20260709 13:14:22.045895  4915 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.104761  4915 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:21.608706  5027 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.088656  4780 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.107959  1601 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:22.108035  4780 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:22.108737  4780 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.089506  4763 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.091220  4879 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:22.099645  5054 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.101941  1597 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 9. Ops: []
I20260709 13:14:22.099658  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:22.102938  5080 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.103807  4657 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.088837  5073 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.107221   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:22.115993   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:22.116104   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.116550   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.116791  4783 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:22.117508   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.117587  5000 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.121059  5027 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:22.138181  5027 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.139056  5057 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:22.097164  5081 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.107378  4915 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.136679  4763 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.137447   853 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:22.137694  1599 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:22.109251  4832 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:22.151507   922 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:22.139576  5073 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.098824   852 rpcz_store.cc:276] 0709 13:14:20.936061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.936242 (+   181us) service_pool.cc:224] Handling call
0709 13:14:20.936687 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:20.937102 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:20.937141 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:20.937695 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:14:21.478361 (+540666us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:21.478424 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.478517 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.083935 (+605418us) raft_consensus.cc:1685] finished
0709 13:14:22.083973 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.084184 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38890,"raft.queue_time_us":855,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":26288,"thread_start_us":445543,"threads_started":1,"wal-append.queue_time_us":716137}
I20260709 13:14:22.125612  4964 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.144434  5055 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.109264  5071 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.167446  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.168694  1593 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:14:22.170820   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63812us
I20260709 13:14:22.111136  5065 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:22.146001  5044 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.165724  5084 ts_tablet_manager.cc:933] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.153790  5080 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.177271  5065 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:22.181138   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 119985us
I20260709 13:14:22.186259   853 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:22.189214  5065 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.160367  5081 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:22.197065   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 81255.3us
I20260709 13:14:22.163712  5035 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:22.150339  4940 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.207880   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.210098   854 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:22.210505  4783 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:22.212857  4881 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:22.214048  4881 raft_consensus.cc:3055] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:22.189188  4766 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.216533  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.221274   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:22.229892  4763 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.214501  4881 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:22.231977  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.233002  4881 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.234454  4928 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:22.240221   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.240365  4928 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.198540  4929 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.242274   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.242993  4928 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.243355  4929 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.236861   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.244282   922 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:22.237594   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.247080  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.237874   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.247861  1002 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.238687   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.248718   923 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:22.239185  5057 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:22.250705  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:22.253538   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52713.7us
I20260709 13:14:22.205408  5005 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:22.245977  4881 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:22.226796  4783 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.255960   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.257076  5073 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.260246  5073 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:22.260659  5073 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.261662  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.262252   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.262423  1599 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:22.262357  4657 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:22.262936   930 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:22.263135  4657 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.265399  4657 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:22.261910   818 connection.cc:256] RPC call timeout handler was delayed by -1.2109546720000708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 13:14:22.267911  4783 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:22.270351   818 connection.cc:256] RPC call timeout handler was delayed by -1.0893460850002157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 13:14:22.270736  4783 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.251128  5057 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.272620   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.273912  1002 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:22.272007   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.275979   922 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.274819   999 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:22.280350  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.214453  4879 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:22.295782  5083 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:14:22.296128  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.297482  5057 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.298763  5044 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.311709   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:22.315550  4860 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:22.316270  4860 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:22.317353  4860 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.320792  5080 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:22.327889   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68461.5us
I20260709 13:14:22.329470   896 leader_election.cc:400] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:22.329789   896 leader_election.cc:403] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:22.329908  5071 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:22.334249   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.348156   999 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:22.349318  4879 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.349803   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.281070  5040 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.353363  5040 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.355273  4940 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.357558   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.358850   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.360177   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.360980   930 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:22.257552  4780 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.375031   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:22.375291   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.378293   927 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:22.380609   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.389977   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.394891  4940 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:22.396723   847 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:22.399677   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.383513  5080 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.402901   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.403858   922 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:22.389817  4879 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.406867  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:22.408870   999 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:22.410876   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.408802  4879 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:22.412760  4879 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:22.413755   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.420950  1599 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12.
I20260709 13:14:22.424494   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.401142  4860 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:22.432672   922 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:22.410580  5040 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.441221   930 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:22.449194  5027 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.449697   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:22.456775   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 127555us
I20260709 13:14:22.457027   968 leader_election.cc:400] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12.
I20260709 13:14:22.457441   968 leader_election.cc:403] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:14:22.455818  5086 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.256s	user 0.002s	sys 0.000s
I20260709 13:14:22.410429  4832 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:22.476290  5088 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.071s	user 0.000s	sys 0.004s
I20260709 13:14:22.481438  4832 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.482867  5081 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:22.483744   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.485481  4928 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12.
I20260709 13:14:22.492789  4993 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:22.493721  4993 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.498371   852 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:22.414914  5065 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.429776  4860 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:22.507037  5027 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.491056  5073 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.509738   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.510051  5080 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:22.510116   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.532974   998 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:22.512217  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.511211   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.502607  5040 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:22.519685   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.538494  1001 raft_consensus.cc:2368] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:22.542774  1593 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:22.545611   930 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:22.557844  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.555884  1000 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:22.563759   997 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:14:22.568611  5085 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:14:22.568799  5065 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.575191   927 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:22.576740   930 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:22.540580  4879 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.584781   922 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:22.587116   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:22.590951   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 219848us
W20260709 13:14:22.592974   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.549767  5045 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:22.593926   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.595089   923 raft_consensus.cc:2368] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12.
I20260709 13:14:22.597781   929 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:22.598222   929 pending_rounds.cc:85] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:14:22.598038  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.598480   929 pending_rounds.cc:107] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 9.6
I20260709 13:14:22.598937  4879 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.600592   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.601258   926 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:22.601792  5045 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:22.602666   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 154039us
I20260709 13:14:22.602718   849 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:22.604871  5040 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.605218   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.598920   929 raft_consensus.cc:2917] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Aborting config change with OpId 9.6: Aborted: Op aborted by new leader
I20260709 13:14:22.598747  1002 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.607591   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.608319   926 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:22.597800   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.610746   929 raft_consensus.cc:2889] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:22.605182  5045 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:22.616691   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.621762   926 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.625567  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:22.626351  5098 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:14:22.627255   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.627410  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.628106   997 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:22.625921   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.629434   924 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.632028   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:58322 (request call id 0) took 5559 ms. Trace:
I20260709 13:14:22.632393  5073 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.633867  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:22.601912  5065 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:14:22.641392   930 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:22.617195  5081 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:22.652802   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.653260  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.659364   998 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.655704   924 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:22.655385   923 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:22.657928   921 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:22.660778   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.634929  4780 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.663187   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:22.661437   928 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:22.632419   929 rpcz_store.cc:276] 0709 13:14:17.072905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.074847 (+  1942us) service_pool.cc:224] Handling call
0709 13:14:17.793902 (+719055us) log.cc:844] Serialized 499 byte log entry
0709 13:14:18.205216 (+411314us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54003282d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b45735 00007f0d07b05d29 00007f0d010841d4 00007f0d01084162 00007f0d01084091 00007f0d0108400c 00007f0d010832c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:18.223568 (+ 18352us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b18001cbf40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:19.044765 (+821197us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b18001cbf40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:19.450918 (+406153us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18001cbf40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa202a6 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d00e0ba20 00007f0d00e1ca29 00007f0d00e1d13f 00007f0d00de12ab
0709 13:14:19.451409 (+   491us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b4c00491fc0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d00e0ba20 00007f0d00e1ca29 00007f0d00e1d13f 00007f0d00de12ab 00007f0d00e0ef3c 00007f0d00e0f3fe 00007f0d0109c50b 00007f0d02a9b381 00007f0d02a9926b 00007f0d02a9fd53 00007f0d02a9fcd9
0709 13:14:22.631889 (+3180480us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8045,"raft.queue_time_us":8960,"raft.run_cpu_time_us":6124,"raft.run_wall_time_us":1233028,"spinlock_wait_cycles":1573535744}
I20260709 13:14:22.666117   999 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:22.649536  4964 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.658799  5070 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.634534  5084 tablet_copy_client.cc:323] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:14:22.683739  1002 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:22.684145   894 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:22.720264   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54952.9us
I20260709 13:14:22.720046  5080 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:22.732113  5045 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.736061   928 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:22.737679   924 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:22.679963  4964 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.739765   853 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:22.740535  4964 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:22.744740   852 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:22.753084   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.732887  5045 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:22.758134   923 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:22.686674  5070 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:22.774369   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.777763   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.780239   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.794021   853 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:22.781843   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:22.794610   926 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:22.797122   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:22.800385   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 110779us
I20260709 13:14:22.805240   852 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.811784   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.804592  5045 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:22.820618   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.829638   926 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:22.821285   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.823338   847 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:22.825753   846 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:22.826440   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.809827  5073 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:22.830646   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.832345   998 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
W20260709 13:14:22.831673   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49934.7us
I20260709 13:14:22.834462  4767 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:14:22.836180  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:22.836474   850 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:22.836810  1599 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:22.839356   848 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:22.839594  1593 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:22.844182   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.839864  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:22.842470  1594 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:22.847352   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.848150   853 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:22.854444   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.866449   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.861428  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:22.872634   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.875363   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:22.876292   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.876592   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.877843   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.878118   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.878568   854 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:22.888711   801 auto_rebalancer.cc:629] Failed to schedule move for tablet cdfea09a06ff4100a0b80619f35f7008: Aborted: Op aborted by new leader
I20260709 13:14:22.889569   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.890796   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.881151   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:22.891909   925 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:22.893061   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.893937   925 raft_consensus.cc:2393] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:22.894836   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.860456  4928 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:14:22.879110   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.897564   854 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:22.883450   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.883061   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.900262   930 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:22.882563   924 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:22.891845   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:22.908674   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.911983   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:22.913406   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.913961   929 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:22.915922   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.918195   848 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:22.918735   927 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:22.941390   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.947184  5005 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:22.948654  5113 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:22.954304   818 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:22.955638   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:22.956727  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 110972us
I20260709 13:14:22.956993   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:22.940436   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:22.969906   930 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.872191  4964 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
W20260709 13:14:22.973088   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 104017us
I20260709 13:14:22.975741   853 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:23.007994   923 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:23.009231   921 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:23.010718  4860 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.011878  1597 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:22.972206  4940 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:23.025995   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 163208us
I20260709 13:14:23.023763  4929 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.028934   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:23.042418  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 83099.7us
I20260709 13:14:23.043680  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:23.044557  4763 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:23.045730   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.049945  5103 log.cc:927] Time spent T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.424s	user 0.001s	sys 0.000s
W20260709 13:14:23.051995   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 97938.5us
I20260709 13:14:23.053226  4928 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:23.054118   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:23.055867   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.057741  4993 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:23.062391  4763 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:23.063464  4763 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.065758  5118 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:23.066910  5118 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:23.067101  5065 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:23.068286  5065 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:23.069239  5065 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.070338  4940 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:23.070547  5065 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:23.071833   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53125.9us
I20260709 13:14:23.073201  5057 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:23.073690  5057 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.074087  5057 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:23.076133  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.083957   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:23.092461   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.079461  5065 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.099687  5065 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.101990  5065 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.103013  4993 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:23.103533  4993 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.103971  4993 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:23.079392  5070 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.109920  5005 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.110709  5005 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.113426  5005 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:23.083071  5096 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:14:23.130297   922 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:23.071081  4940 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.134050   852 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:23.135560   929 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
W20260709 13:14:23.140128   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67461.3us
I20260709 13:14:23.054158  4928 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:23.147802   930 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
W20260709 13:14:23.149544   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 95478.3us
I20260709 13:14:23.150825   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:23.151253  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 69889.4us
I20260709 13:14:23.151722   851 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:23.151979  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:23.137652  5035 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.154711  5080 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:23.155953  5080 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:23.156356  5080 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:23.157032  4940 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:23.157215  5080 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.158206   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.158442  5080 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.159029  5044 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.149174  4928 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:23.160794  5044 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.160574  4928 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.161659   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:23.162267  4928 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.163074  5044 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:23.163468  5044 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.163924  5044 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:23.164228  4860 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.164354   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.169991  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:23.171845   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 87810.4us
I20260709 13:14:23.172670   896 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:23.177800  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52040.7us
W20260709 13:14:23.180058  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 108743us
I20260709 13:14:23.210871  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:23.153542  5113 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.119892  4993 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.218876   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.234272  4780 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:23.234910  4780 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.235671  5115 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:23.236311  5115 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.238978  5115 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.239979  5122 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.243182  4832 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:23.243669  5119 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.244262  5119 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.244606  4832 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:23.244680  5119 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.247949  4993 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.175294  4767 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:14:23.229522  5121 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:14:23.250624  5121 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:23.218318  5070 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.252063  4830 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:23.252535  4830 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:23.232443  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:23.227237  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 46196.1us
I20260709 13:14:23.251194  5121 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
W20260709 13:14:23.261157   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.263057  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:23.269307   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.179101  5035 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:14:23.263657  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.298105  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:23.302377   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.302850  1599 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:23.298288  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:23.306824  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:23.307762   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.257166  5065 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:23.315840  1596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543"
dest_uuid: "597687e464174d1ea6043161e037d240"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:41040
I20260709 13:14:23.316044  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:23.316645  1596 raft_consensus.cc:606] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.312374   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:23.306916  1593 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
W20260709 13:14:23.312350   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 139750us
I20260709 13:14:23.256175  4830 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.264292  4993 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:23.319893   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 143973us
I20260709 13:14:23.320188  4830 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.321133   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:23.326370  5121 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.327800  5121 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.312740  1001 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:23.313436  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.331132  1594 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:23.333074   896 leader_election.cc:400] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:23.333570   896 leader_election.cc:403] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:23.333645  1000 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:23.336694   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.338940   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:23.339862  5080 raft_consensus.cc:3060] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:23.340324  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:23.341368  4881 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.334739  5045 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.342778  5045 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:23.343745  5045 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.345932  5065 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:23.347009  5065 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.351432  5080 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:23.353657  5080 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.355924  5073 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:23.365202   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet cdfea09a06ff4100a0b80619f35f7008 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:14:23.365370   894 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:23.365942   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet cdfea09a06ff4100a0b80619f35f7008 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-cdfea09a06ff4100a0b80619f35f7008
I20260709 13:14:23.366251  5080 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.366989  5080 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:23.368816   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 199730us
I20260709 13:14:23.370059   895 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.373008  1599 raft_consensus.cc:2468] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:23.375907   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:23.381462   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.384274  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.375952   929 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:23.390906   928 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:23.391045   894 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.369347  4929 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:23.440301  4929 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.442353  5057 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.372612  5080 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.448835   996 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:23.375186  5035 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.446070  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:23.317687  5115 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:14:23.480749  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:23.505050  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 250371us
I20260709 13:14:23.495791  5122 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:23.511363  5122 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:23.517412   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 353163us
I20260709 13:14:23.518455  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.498101  5035 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.506608  5070 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.529774  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.522428   819 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:23.533669   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 492248us
I20260709 13:14:23.535913  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.536614  1000 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:23.539162   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 228716us
I20260709 13:14:23.529323  5070 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:23.542919   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 202733us
I20260709 13:14:23.547380   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:23.546658   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 165312us
I20260709 13:14:23.544305   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:23.537807  5035 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:23.554054   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:23.554623   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 163675us
I20260709 13:14:23.555230   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:23.556001   921 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:23.555455   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:23.555447   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:23.543128  4763 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.560808   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:23.566996  5057 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:23.568081  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.570767  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.571309   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:23.588058  1004 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:23.571919   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:23.572532   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:23.573050  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.573525   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:23.574698   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.574914  5130 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:14:23.576179   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.576819  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.543447  4879 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:23.584715   999 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:23.547259  5070 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.586036  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:23.586383   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.586776  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:23.596521   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44660.6us
I20260709 13:14:23.598708  1597 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:23.600265   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:23.547859  5115 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:14:23.590690  5128 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:14:23.607295  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
W20260709 13:14:23.608263  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.609541   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.591444  4993 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:23.558658  5138 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.553364  5044 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.614627  5149 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.603685  5135 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:23.613018  4993 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.611377  4763 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:23.620433  5057 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:23.630113  4993 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:23.636581   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:23.637324  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60649.2us
W20260709 13:14:23.649835   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.629158  4763 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.629797  5135 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:23.655956   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 83538.8us
I20260709 13:14:23.636549  5138 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:23.639729  5147 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.721091  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60900.8us
I20260709 13:14:23.726835  5151 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:23.728648   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 92156.8us
I20260709 13:14:23.639323  4879 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:23.654510  4763 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:23.634474  5146 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
W20260709 13:14:23.738204   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 151863us
W20260709 13:14:23.747722   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 109030us
I20260709 13:14:23.740361   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:23.655046  5142 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:23.729663  5130 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:14:23.660889  5138 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:23.652572  5137 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:14:23.756496  5000 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:23.763090  5144 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.763357  5000 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:23.763949  5144 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:23.764320  5000 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.765285   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.735109  5147 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.768146  5136 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:23.767967  5135 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.769506  5136 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.769948  5136 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:23.771631  5129 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:14:23.772136  5057 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.772555  5057 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:23.772950  5057 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:23.775936   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.777225  5139 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.778052  5139 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:23.779735  5134 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:14:23.781001  5134 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:23.781404  5134 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:23.782462  5134 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:23.786154  5119 consensus_meta.cc:287] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.005s
I20260709 13:14:23.786556  5119 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.737761  5148 raft_consensus.cc:2804] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:23.772887  4879 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.799764  5115 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.762954  5152 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:14:23.752853  5121 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:14:23.793656  5146 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:23.802791  5119 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.803808   928 raft_consensus.cc:2468] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
W20260709 13:14:23.804302  5150 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:14:23.806936  5151 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:14:23.758309  5138 raft_consensus.cc:3060] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.832827  1000 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:14:23.822288  4860 consensus_meta.cc:287] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.003s
I20260709 13:14:23.837268  5134 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:23.841560  5143 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:14:23.846719  4763 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.848127  5154 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:23.848930  5155 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:23.835289  5148 raft_consensus.cc:697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:23.807729  4993 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:23.840939  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 79652.9us
I20260709 13:14:23.876170  4860 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:23.822535  5115 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:23.890640   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 124235us
I20260709 13:14:23.891033  1597 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:23.893095   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 140320us
I20260709 13:14:23.894541  1596 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:14:23.856976  5149 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:23.895339   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:23.900173   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 157982us
I20260709 13:14:23.898286   969 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:23.885536  5044 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.904577   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:23.911057  5156 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:23.911168  5133 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:23.911748  5156 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:23.912181  5156 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.913059  5133 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.913606  5133 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:23.914673  5133 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.915696  5137 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:23.893102  5136 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:23.850477  5146 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.897481  5121 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.923714   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.925653   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.929533  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.910014  4929 raft_consensus.cc:2804] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:23.879619  5154 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.929204  5035 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:23.885989  5155 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.890966  4860 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.930632  5121 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.947829  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:23.961091  1594 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:23.947532  1004 raft_consensus.cc:606] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.835947   741 maintenance_manager.cc:643] P ad0b4d62f81540a8878fe1d1c1552895: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 5.512s	user 0.150s	sys 0.066s Metrics: {"bytes_written":29069,"cfile_init":1,"dirs.queue_time_us":176359,"dirs.run_cpu_time_us":548,"dirs.run_wall_time_us":62088,"drs_written":1,"lbm_read_time_us":97251,"lbm_reads_10-100_ms":4,"lbm_write_time_us":14568,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":28,"peak_mem_usage":0,"rows_written":63,"thread_start_us":97932,"threads_started":1}
I20260709 13:14:23.962091  1004 raft_consensus.cc:614] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:23.969115   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 43603.7us
I20260709 13:14:23.971073   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:23.900985  5080 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.927390  5027 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:23.983826   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 68779.9us
I20260709 13:14:23.944780  5115 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.948567  4763 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.947324  4993 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:23.953428  4860 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:23.979326  5121 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.905124  5149 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.993969   742 maintenance_manager.cc:419] P ad0b4d62f81540a8878fe1d1c1552895: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9138 bytes on disk
I20260709 13:14:23.994650  5146 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.943074  5147 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.969909  5136 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:14:23.997354  5156 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.998307  5151 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:24.000475  5148 consensus_queue.cc:237] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:23.961596  4929 raft_consensus.cc:697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:24.001031  5057 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.987800  5027 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:24.006597  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 81498.1us
I20260709 13:14:23.945742  5155 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:23.942888  5154 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.015285  1594 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:23.981195  5035 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:24.064292   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79505.2us
W20260709 13:14:24.075018   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 117227us
I20260709 13:14:23.935530  5070 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:23.977433  4832 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:14:24.038058   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.041093  5138 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.049976   853 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:24.054152   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82902.1us
I20260709 13:14:24.077427   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.058611   855 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:24.058889   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.061447   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.058812   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.079547   852 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:24.080403   896 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.080772   846 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.076886   741 maintenance_manager.cc:643] P ad0b4d62f81540a8878fe1d1c1552895: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.033s	user 0.002s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":480,"lbm_reads_lt_1ms":4}
I20260709 13:14:24.088344   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:24.091975   847 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:24.063716  5164 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:24.096415   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.096475   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.100473  1002 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:24.102082   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.102662   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.084008  5146 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:24.100907  1002 pending_rounds.cc:85] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 4
I20260709 13:14:24.108974   852 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:24.102975   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.110216  1002 pending_rounds.cc:107] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:24.110626  1002 raft_consensus.cc:2889] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.103322   995 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:24.092253  5081 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:24.077994  5134 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:24.108705   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.114941  5057 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.115731  5169 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:24.098621  5144 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:24.135188  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.135917  1001 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:24.112481  5035 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:24.136410   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.136978  5035 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.135851  4929 consensus_queue.cc:237] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.113435  5137 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:24.144888   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.121654  5081 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.152707  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:24.153633  1594 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:24.146222  5137 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:24.136365  4860 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.141557  5146 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:24.150431  5081 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:24.146821  5169 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.190405   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.135174  5160 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:14:24.192652  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.196630   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.152777  5080 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.194672   999 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:24.153466  5176 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:24.181488  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.181113  5136 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.181514  5137 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:24.183390  5142 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.203078   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:24.235802   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.186959   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 864581us
W20260709 13:14:24.224941   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 122012us
I20260709 13:14:24.221714   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:24.199582  4763 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.244509  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.247721   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:24.204293  5177 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.202227  5176 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.262562  1002 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:14:24.255175  5169 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.257862   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.254637  5157 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:24.197527  5146 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.222005  4832 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:14:24.272480  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:24.236483  5134 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:14:24.222872  4860 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.248584  5156 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.251425  5027 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.259965  5070 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:24.301983  5178 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:14:24.242913  5136 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.303169   995 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:24.266762  5057 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:24.267481  5176 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.247859  5154 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:14:24.303009  5184 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:14:24.314057   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 92423.1us
I20260709 13:14:24.319410  5167 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:24.350843   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.295738  5157 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:24.325774  4860 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:24.327847   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 88174.1us
I20260709 13:14:24.336898   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:24.337821   995 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:24.338783  1002 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:24.340004  5186 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:24.273283  5138 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:24.343744   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.344249   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 99705.8us
I20260709 13:14:24.323828  5144 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:24.357723   896 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.373812   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:24.374200   940 tablet_copy_source_session.cc:215] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:24.375252   940 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:46362 (request call id 614) took 1109 ms. Trace:
I20260709 13:14:24.375949  5151 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.375521  5176 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.376648  5151 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:24.377441  5151 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:24.377807  5151 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:24.377846  5119 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:24.378284  5151 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.378371  5176 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.378953  5073 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:24.359898  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:24.380455  5073 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:24.380208  5174 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:24.380851  5073 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:24.360726   967 leader_election.cc:400] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12.
I20260709 13:14:24.361375   818 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:24.381235  5174 consensus_queue.cc:1243] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:24.353909   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:24.381996   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.382200   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:24.381880  5073 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.383270  5073 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:24.340358  5057 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:24.393005   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.392858   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.367007   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.368888  5081 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.267002  5177 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.370920  5146 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.361372  5070 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:24.381062  1593 raft_consensus.cc:2368] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:24.362490  5148 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.383939  5168 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:24.383754  5147 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:24.384485  5156 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:24.385015  5154 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:24.360848  5167 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:24.420935   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.412209   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:24.389530   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.397819   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.398006   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.399681   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.400292   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.408442   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.394899   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.395419  1001 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:24.371843  5144 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:24.408130   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.455396   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.463954   926 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.464113   928 raft_consensus.cc:2376] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14. Re-sending same reply.
I20260709 13:14:24.465117   927 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:24.466312   921 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 13: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:24.396047  5057 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.378320  5119 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:24.478258   930 raft_consensus.cc:2410] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:14:24.424773   922 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:24.400781  5138 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.415949  5131 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:14:24.491922  5156 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:24.493062  5156 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.495846  5045 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.496668  5045 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.498972  5045 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.499102  5194 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.499811  5194 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.502161  5194 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.399370  4881 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:14:24.505811   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.507021   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.507125   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.508492   928 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:24.479449  4832 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.455238  5146 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.507988   921 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:24.515564   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.510767   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.516637   852 raft_consensus.cc:2368] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:24.408587  5189 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:24.520328   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.467828  5168 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:24.517244   930 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:24.465529  5169 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.533193   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.412472  5176 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:14:24.466535  5154 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:24.507143  5144 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:24.522424   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 110293us
I20260709 13:14:24.389797   940 rpcz_store.cc:276] 0709 13:14:23.333302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.364938 (+ 31636us) service_pool.cc:224] Handling call
0709 13:14:24.373852 (+1008914us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54003282d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b493f7 00007f0d07adf036 00007f0d07ae8a3e 00007f0cff3ac27b 00007f0d07ae89ba 00007f0d07ae3acc 00007f0d07adec7b 00007f0d07ad38d1 00007f0d06b874b4 00007f0d06b87442
0709 13:14:24.375153 (+  1301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209187584}
I20260709 13:14:24.522868  5188 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.522934  5136 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.396171  4860 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.533980   930 raft_consensus.cc:2410] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:24.421392  5148 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.517954  5138 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.535184   926 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:24.519074  4832 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.543776   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:24.533046  4929 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.537197  5189 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:24.558647   896 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:24.552484   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.521152  5146 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.569356  5168 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.570564  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:24.570982  5146 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.571139  1001 raft_consensus.cc:606] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:24.571579  1001 raft_consensus.cc:614] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:24.555035  4832 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.573542  5138 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.573582  5167 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.574530  5040 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:24.575050  5040 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:24.575456   852 raft_consensus.cc:1275] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:24.575855   852 pending_rounds.cc:85] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:14:24.576191   852 pending_rounds.cc:107] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:24.576463   852 raft_consensus.cc:2889] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.575917  5040 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.576753  5000 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:24.557853   924 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:24.577116  5040 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:24.574103   930 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:24.547466  5119 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.558157  5137 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.545768  5147 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:14:24.537719  5169 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.567039  5149 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.596753   850 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:24.567622  5148 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.549731  4860 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.568236  5045 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:24.568518  5070 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:24.553150  5160 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.574451  5081 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.552734   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.613335   926 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
W20260709 13:14:24.583663  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 194264us
I20260709 13:14:24.589869  4832 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:24.590829   852 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:24.591516  1000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51144 (request call id 2036) took 1050 ms. Trace:
I20260709 13:14:24.614596  1567 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:24.592113  5189 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:24.594843   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51331.1us
I20260709 13:14:24.606576   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.620072   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.611025  1001 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:24.624912  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.625659  1003 raft_consensus.cc:2393] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:24.615477  5045 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.615787  5070 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:24.616557   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.617527   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.631369   925 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:24.615793 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:24.626303   968 leader_election.cc:400] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:24.626929   930 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:24.629690   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.634857  1001 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:14:24.634299   850 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.634642   968 leader_election.cc:403] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:24.619793  1000 rpcz_store.cc:276] 0709 13:14:23.541166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.541451 (+   285us) service_pool.cc:224] Handling call
0709 13:14:23.545279 (+  3828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:23.835169 (+289890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.591224 (+756055us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400088ed4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:24.591426 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660630400}
I20260709 13:14:24.635231  1001 pending_rounds.cc:85] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 9
I20260709 13:14:24.636663  1001 pending_rounds.cc:107] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:14:24.637104  1001 raft_consensus.cc:2889] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.633213   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.636325   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.641180   850 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:24.548753  5188 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.647534   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.647915  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:24.655184   854 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.638688  5122 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:24.639689  5195 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.646426  5119 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:24.668655   852 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:24.681126  4860 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.652737  5131 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:24.682723   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.683511   854 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.684327  5169 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.672065  4780 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.686177  5081 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:24.686769  5081 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:24.688773   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.630661  5160 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.712041   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.637758  5154 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.716439   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.668709  5122 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:24.717717   930 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:14:24.719807   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.720793   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.722818   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.726032  5081 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.727090   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.727890   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.728742   925 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 8: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.743059  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:24.744112  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:24.745072  5141 raft_consensus.cc:993] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:24.760264  1593 raft_consensus.cc:2393] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 7: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:24.731382  5035 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.774011  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:24.767848   924 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:24.773945  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:24.765874  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.773973   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:24.756069   821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.855s (SENT) [suppressed 18 similar messages]
I20260709 13:14:24.771761   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.780090   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:24.785019  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.788501   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:24.789124   924 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.790731   853 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.787469   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.793036   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.798156   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.799921   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.765232  4763 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:24.768163  5081 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.803038   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.806392  1004 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:24.807320  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:24.807468   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 51454.2us
I20260709 13:14:24.768069  5065 raft_consensus.cc:1081] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:24.808822   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.773296  4832 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:24.791947  4940 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.817778  1599 raft_consensus.cc:2393] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 5: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:24.817365   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.696803  5070 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.825766  1000 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:24.831641   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:24.757741  5151 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.795368  5168 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:24.774715  5146 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:14:24.808820  4763 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:24.808703  5121 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:24.841015  5171 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:14:24.847345   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.823563  5137 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:24.864498   921 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:24.861253  4763 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.812988  5174 consensus_queue.cc:1048] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:14:24.872138   924 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:24.875689   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.878322  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.889655   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.826313  5142 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:14:24.822306  5081 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.835788  4832 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:24.898882  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.899746   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.868238  5168 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:24.847805  4940 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.873065  5080 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.886080  4881 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.885293  5045 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.911692  5182 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:24.913166  5204 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:14:24.924448  5207 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 13:14:24.964056  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:24.879931  5160 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:24.932128  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 151913us
I20260709 13:14:24.876366  5216 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:24.944957   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 97711.1us
W20260709 13:14:24.946712  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 130717us
I20260709 13:14:24.968439   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:24.910943  5196 consensus_queue.cc:1059] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:14:24.844089  5122 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.973553  5045 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:24.973997  5045 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:24.975627  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.976279  1000 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:24.942806  5073 consensus_meta.cc:287] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 13:14:24.977376   926 raft_consensus.cc:2410] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:14:24.910046  5149 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.966854  1564 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:24.979983  5177 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.980387   851 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:24.980609   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:24.970371   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.972057  5084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdfea09a06ff4100a0b80619f35f7008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:24.972210  5081 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.987674  5081 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:24.988051  5081 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.953069  4881 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.988536  5161 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:24.990070  5161 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:24.991634  4881 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:24.983438  4940 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:24.952118  5035 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.948506  5080 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:24.988965  5081 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.986027   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.026717   895 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.027377   924 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.004894  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:25.027750   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.961993  5142 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.951674  5219 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.952272  5188 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.014012  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.007128  5065 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:25.026813   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:24.979274  5057 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.026163  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.029722   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.032407   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:25.025050   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:24.996428  5161 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.034586   929 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:25.039628   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.038218   925 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:25.034622  1601 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:25.013849  4875 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:24.988632  5151 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.009289  5149 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:25.041344   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.042999   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.024997  5160 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.044757   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.047101   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.048255   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.048754  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.052711   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.053308   924 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:25.033725  5148 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.057943   998 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:25.058202   894 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.037856  5065 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.062141  1601 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:14:24.967760  5182 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.074999   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.043557  5214 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:25.050963  5161 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.033025  4881 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:14:25.084539  5223 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:25.085407  5225 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:14:25.060194  5151 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:25.046639  5146 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:25.033984  5142 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.093004   921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:46362
I20260709 13:14:25.091472   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.092514  5195 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:25.049654  4875 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.094319  5167 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:25.087986  5005 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.083034  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.097010  1597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 690) took 1167 ms. Trace:
I20260709 13:14:25.079593  5065 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.100452   925 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:25.036863  5219 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.090835  5149 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.105686  5142 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.105932   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.106566   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.059514  5160 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.111590  5157 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.112632  5057 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.115165   851 raft_consensus.cc:1275] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:14:25.050076  5168 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.116871   929 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 13:14:25.126093  4929 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.000s
I20260709 13:14:25.120321  5146 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.147547  5080 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.147876  4929 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.148048  5162 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.149636  5146 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.149678  5162 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.151221  4875 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.139838  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.152865  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.153450  1003 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:25.153450  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.154291  1002 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:25.154536  5162 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.151078  1004 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:25.158653  5151 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:25.159955  5146 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:25.160455  5146 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:25.161616  5146 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.162981  5146 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:25.165289  5224 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:14:25.166040   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.121658  5005 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:25.170827  5005 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.171248   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.172214  4832 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.126819  5119 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:25.176862  5119 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.177316  5119 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.177942  5080 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:25.178503  5080 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.129822   850 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:25.129263   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.182719   999 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:25.183796  5080 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:25.184405   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.186161  5216 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:25.132858   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.134815   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.187049   849 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:25.136795   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45388.8us
I20260709 13:14:25.138331   929 pending_rounds.cc:85] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:14:25.188356   929 pending_rounds.cc:107] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:25.188618   929 raft_consensus.cc:2889] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.188953   925 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.190982   929 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.118397  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.193060   929 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.120585  1597 rpcz_store.cc:276] 0709 13:14:23.928444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.941574 (+ 13130us) service_pool.cc:224] Handling call
0709 13:14:23.941971 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:23.955226 (+ 13255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:23.955257 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:23.955747 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.035757 (+ 80010us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:24.035821 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.090867 (+1055046us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5400464bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:25.090920 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.090967 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:25.090981 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.096061 (+  5080us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1142,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15151,"spinlock_wait_cycles":29558016,"thread_start_us":72476,"threads_started":1,"wal-append.queue_time_us":156273,"wal-append.run_cpu_time_us":2200,"wal-append.run_wall_time_us":1006046}
I20260709 13:14:25.194721   929 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:25.197860   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.122749   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.144268  5219 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.146268  5160 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:25.206427   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.128749  4767 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:25.166868  5218 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:14:25.168731  5182 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.160571  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.120199   921 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:25.213243   921 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.213816  1002 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:25.170065   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.179255  5045 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.200361  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.142153  5136 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:25.163902  4763 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.205927  1000 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:25.120146  5167 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:25.207412   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.208221   821 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.211733  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:25.214824   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44834.4us
I20260709 13:14:25.214854   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.216311  5045 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.228822   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.221767   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.241453  5155 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:25.242394  5155 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:25.234745  5162 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:14:25.242759  5155 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.243714  5155 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.244944  5155 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.218645  5148 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:25.218477  5136 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.247849  5223 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.248605  5223 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:25.248445  5136 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.249478  5136 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.249814  5223 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.251904  5148 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.247852  5214 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.216774  5196 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.252453  5148 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.258147  5005 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.247067  4875 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:14:25.260555  5229 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:14:25.263047  5144 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.235205  5226 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:14:25.265899  5160 consensus_queue.cc:1048] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:25.267043  5170 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:25.267472  5170 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.240450   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:25.232928  5232 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:25.240624   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.240804   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:25.234911  5167 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:25.272688   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 74906.9us
W20260709 13:14:25.270468   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.273584   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:25.279989  4832 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:25.280334   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58626.5us
I20260709 13:14:25.280951  5162 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:25.281528  5162 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.281930   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.283252   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.283631   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.284479   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.273102  5035 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.289422  4767 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.289907  5176 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:25.291781  4763 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.293824  5148 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.294775  5115 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:25.295311  5233 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.057s	user 0.002s	sys 0.001s
I20260709 13:14:25.212030  4879 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:14:25.301729   921 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.217654  5081 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.306857   930 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:25.307157  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.311298  5142 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:25.311429  1000 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:25.311810  5142 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.312259  5142 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:25.317487   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.289459  5162 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.296169   818 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.295964  4780 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:25.320333  5084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:37790 (request call id 685) took 3225 ms. Trace:
I20260709 13:14:25.308964  5176 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:25.319046  5217 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 13:14:25.320792  5084 rpcz_store.cc:276] 0709 13:14:22.094578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.096439 (+  1861us) service_pool.cc:224] Handling call
0709 13:14:22.190366 (+ 93927us) ts_tablet_manager.cc:934] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
0709 13:14:24.744630 (+2554264us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:14:25.115524 (+370894us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:25.120088 (+  4564us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:14:25.295448 (+175360us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:14:25.319734 (+ 24286us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:25.320180 (+   446us) ts_tablet_manager.cc:989] Replica registered.
0709 13:14:25.320246 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":30303,"tablet-copy.queue_time_us":68381,"thread_start_us":186469,"threads_started":2}
I20260709 13:14:25.322422  5084 tablet_copy_client.cc:806] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:25.305938   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:25.303799   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 63066.7us
I20260709 13:14:25.326838   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:25.323374  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 354056us
I20260709 13:14:25.324327  5081 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.345422   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.349769  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.300951  5168 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:25.355758   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.350405  1601 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:25.363361   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:25.365417   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 69312.5us
I20260709 13:14:25.366376   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:25.368583  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.370437  1596 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:14:25.376235   922 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.379454   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.333154  5149 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:25.327450  5035 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:25.385596   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.348554  4929 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.397604   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.399133   969 leader_election.cc:400] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8.
I20260709 13:14:25.393019   995 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.389176   929 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:25.363181  4767 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:25.363054  5057 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.362115  5168 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:25.378494  5195 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.385340  5005 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.406760  1000 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:25.404686   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:25.381423  5115 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.387020  5084 tablet_copy_client.cc:670] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
W20260709 13:14:25.414150   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.441143  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.401338  5179 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.406958  5134 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:25.411897  5168 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.456779  1601 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:25.458926   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 54327.5us
I20260709 13:14:25.460011   995 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:25.464820   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 98523.7us
I20260709 13:14:25.475817   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.387329  5170 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.476727   928 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:25.477108  5170 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.477680   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.478114  5005 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:25.478547   928 raft_consensus.cc:2410] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:25.479379   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.478137   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.484177   927 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.484570   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.482631   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.485283   928 raft_consensus.cc:2368] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:25.486182   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.489161   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:25.497603  5236 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:25.500465   927 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:25.504170   927 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:25.487771  5179 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.505606   927 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:14:25.505867  5148 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.505707  5179 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.482877  5065 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.501529  5005 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:25.502647   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:25.459965  5238 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:25.468462  5198 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.515794   927 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:25.468689  4929 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.475040  5231 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
W20260709 13:14:25.516314   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 49765.7us
I20260709 13:14:25.511540  5134 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:25.472163  5171 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:14:25.519302  4928 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:14:25.526723  4780 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:25.530948  4780 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.531476  4780 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.534803  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.535431  1004 raft_consensus.cc:2376] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10. Re-sending same reply.
I20260709 13:14:25.536373  4940 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:25.517885  4929 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:25.542693  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.549337   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.550944   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.548990   998 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:25.527922  5134 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:25.557790  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.562076  1002 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:25.540175  5119 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.520166  5080 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.534503  5238 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.567166  5057 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:25.545804  5137 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.556192  4879 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:14:25.545558  5188 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:25.557723  5005 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.568718  5065 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.548655  4929 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.571328  5155 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.587924   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.600612   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.590853   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.601794   853 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:25.556440  5196 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.586675  4875 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.575274   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:25.598730   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 108450us
I20260709 13:14:25.519411  5198 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.571790  5134 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.591069  5243 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.610636   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.616027   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.618801   850 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.622011  5153 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.597072  5137 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.606559  5155 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.624086   853 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.624969   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.628459   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.608268  5167 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.608053  4875 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.635270   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.636646   850 raft_consensus.cc:1240] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:25.641340  5155 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.642073   850 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:25.614794  5243 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.645275  5198 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.645955  5243 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.622565  5153 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.643409   850 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:14:25.649508   854 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:25.654830   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.652400   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.656945   853 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:14:25.659670  5244 log.cc:927] Time spent T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:14:25.656186   848 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:25.633960  5238 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.676644  5080 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.688915  4780 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.693740   851 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.692529   849 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.696144  5153 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.697000   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.680127   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.699980   850 raft_consensus.cc:1275] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:14:25.701200   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.703038  5137 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.704316   849 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:25.709761  4875 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.714919   921 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:25.719067  5246 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:25.720309  5142 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.722577  5142 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.727519  5136 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:25.728201  5136 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.732890  5168 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.734151   852 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:25.735211  5168 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:25.736238   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 381944us
I20260709 13:14:25.812584  5176 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:25.813926  5176 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:25.814380  5176 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.736830  5070 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.816337   926 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:14:25.816653  5070 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.817996   924 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:25.739857  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.818915  5070 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:25.819288  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79526.4us
I20260709 13:14:25.820245  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.820444  5131 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.821570  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:25.822146   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.822289  5194 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:25.740401  5223 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.822788  5194 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.823280  5194 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:25.823808   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.824368   927 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:25.741075  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:25.825188  1594 raft_consensus.cc:3055] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:25.825455   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.745213  5148 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.826886   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.827009   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.827498   848 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.827278  5148 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.825479  1594 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:25.828583  1594 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.829387  5148 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.829629   855 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.830148   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.747929   921 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:25.745792   925 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:14:25.746493   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.832805   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.833309   925 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.743522  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:25.836138  5220 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:25.836671  5220 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:25.837071   927 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.838400  5065 consensus_queue.cc:1059] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:25.747978   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.749261   930 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:14:25.749604   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.840003   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.729421  5136 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.844659  5136 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.844952   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.845176   925 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:25.845687   930 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.845896   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.846630   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.847159   927 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.848392   922 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:25.848660  4881 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:25.849611  4763 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:25.849951  4763 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:25.850252   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.850777  4763 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.851505  1594 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.852756  5084 tablet_copy_client.cc:538] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:25.854600   926 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:14:25.738528  4929 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.770092  5167 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.776535  5241 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:14:25.849972   968 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85bbb1b886474a538a2f1a32dd91b560. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.756808   852 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:25.784724  1004 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:25.857800  4929 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.859196   928 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:25.859449  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:25.860239   896 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:25.860914  4763 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.863247   818 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.864058  5113 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:25.864332  5243 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:25.865002   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.865705  5243 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:25.866074  5243 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.866434  5243 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.866328  4993 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:25.866561  5113 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:25.867408  5113 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.868476  5113 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.868377  5174 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.871086  5174 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:25.784968   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:25.871963  5174 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:25.872658   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 87802.1us
I20260709 13:14:25.873536   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.802583  5157 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.782861  4780 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.823000  5223 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.825519   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.896179   923 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:25.837525  5220 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.846065   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.855986  5241 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.782596  5195 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:25.944278  5195 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:25.945698   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.945444  5195 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.946645  5167 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:25.946815  5195 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.947170  5167 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.947723  5167 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.948714  1595 raft_consensus.cc:1240] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:14:25.948797  5148 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.949507  5148 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.950503 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:25.856334   997 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:25.921640  5134 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.953459  5134 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.955950  5148 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.873528   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:25.957070   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83623.8us
I20260709 13:14:25.959789  5134 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:25.959586  5194 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.960745  1597 raft_consensus.cc:1240] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:25.786273   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.871479   928 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:25.803571  5045 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:25.963773  5045 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:25.964679  5045 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.965286  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.965638  5045 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.966064  1001 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.876000   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:25.876313   926 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:25.968035   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.969100   926 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:25.876915   848 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:25.877166   922 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:14:25.970779   922 pending_rounds.cc:85] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:14:25.971088   922 pending_rounds.cc:107] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:14:25.971328   922 raft_consensus.cc:2889] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.972083   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:25.750262   850 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:25.972806   997 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.878206   924 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:14:25.973451   924 pending_rounds.cc:85] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 5
I20260709 13:14:25.973716   924 pending_rounds.cc:107] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:25.880648   967 leader_election.cc:400] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:25.973970   924 raft_consensus.cc:2889] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.974261   967 leader_election.cc:403] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:14:25.974912   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94409us
I20260709 13:14:25.975775   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.977807   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.979027   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:25.981624  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.982299  1600 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:25.985292  1004 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:25.988509  5194 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:25.676272  5119 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:25.895272  1594 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:25.898339   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:25.998379  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:26.000958   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 102677us
I20260709 13:14:25.906963   855 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:25.910470   930 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:25.912297   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:25.917882  5223 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:25.918435  5168 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:25.921578   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:25.904141  5170 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.923345   851 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:25.819710  5176 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:25.926915  1598 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.928941   925 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:25.927496  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:25.932052  1599 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:25.899276  5065 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.936038  5219 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:25.899449  5155 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:14:25.943779  4780 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:25.943087  5241 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:25.990674   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:25.881003   927 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:25.996524  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:25.997651   999 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:26.012332  1596 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:26.000945  1004 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:26.045230   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:26.032619   895 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:26.050906   847 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:26.015100  4879 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:14:26.052980   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 62387us
I20260709 13:14:26.055794   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:26.052795  1000 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:26.019120  1001 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:26.010988  5170 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:26.057488   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:26.060777  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:26.016029  5187 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.027561  5035 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:14:26.062024   922 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:14:26.014329  5177 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10.
I20260709 13:14:26.063740   819 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:26.064739  1564 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:26.072093   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 49077.6us
I20260709 13:14:26.076771   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:26.050560  5115 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:26.022893  5136 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:26.021849  4929 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:14:26.059618  5148 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:26.021616  5065 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.064692  5070 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:26.101251  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:26.078457  4881 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:26.103626  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.054241  5168 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:14:26.063236  5170 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:26.052901  5137 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:14:26.178642  5261 log.cc:927] Time spent T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:26.075917  5151 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:26.053613  4780 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.155225  5167 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.181686  4780 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:26.182160  5167 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.175034  4881 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:26.092978  5115 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:26.183954  4881 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.184423  5115 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.186064  5115 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.111409  5156 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:14:26.111794  5148 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:26.162052  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.081629  5027 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:14:26.188597  5027 consensus_queue.cc:1243] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:26.105332  5238 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:26.189209  5238 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:26.189640  5238 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:26.196130  5263 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:14:26.167223  5070 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:26.087798  4763 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:26.176885  1593 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:26.198853  5148 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:26.212636  5259 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:14:26.214329  5196 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:26.114246  5174 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 13:14:26.108927  5055 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:14:26.228708   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.228447  5243 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.109303  5005 consensus_queue.cc:1059] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 13:14:26.172361   997 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:26.252628  4780 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:14:26.271867  1564 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:26.275695   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 215331us
W20260709 13:14:26.277352   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 160737us
I20260709 13:14:26.248092  5151 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:26.277479   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 319258us
I20260709 13:14:26.173372  5162 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:14:26.288290   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:26.173171  5188 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:14:26.172766  5131 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:26.314546  5005 raft_consensus.cc:3055] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:26.315119  5005 raft_consensus.cc:740] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:26.316500  5223 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:26.316162  5005 consensus_queue.cc:260] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.316623   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:26.317044  5223 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.317528  5223 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.317595  5142 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:26.318193  5142 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.318671  5142 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.314582  5151 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.317605  5005 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:26.278903  4940 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:26.322214  4940 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.325901  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:26.327196  5223 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.329890  5223 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.331926   921 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:26.331756  5142 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.334148  5142 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.327730  4940 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:26.336994  5055 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:26.337545  5055 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:26.337101   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:26.338798  5243 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.339432   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:26.339672  5141 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:26.339798   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.340148  5141 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.340745  5141 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.342578   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.343117  5162 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:26.340140  5065 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.343932  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.344349  4940 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:26.344540  1001 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:26.344930  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.345216  4940 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:26.345813  4940 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:26.346078  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.342067   896 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:26.347493  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.346783  4940 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.344298   997 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:14:26.350950  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:26.351275   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.288208  4763 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:26.348213  5157 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:26.357792   998 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:26.357945   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.346714  1004 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
W20260709 13:14:26.296687  5260 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:14:26.353410  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.285890  5177 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:14:26.355046  1000 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:26.348116  1003 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:26.288998  5196 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:26.338413  5055 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.300132  5134 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.292711  5226 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:14:26.369227  1000 raft_consensus.cc:1240] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:26.371521   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.360010   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:26.293704  5168 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.375342  1002 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:26.377501   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.378669   969 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:26.380697   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 204819us
I20260709 13:14:26.380885   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.383152  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.384408  1001 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:14:26.390722   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.393564   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.358901  5073 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:26.397374  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.398017  1003 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:26.398566   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.363461  5005 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.371085  4940 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.365927  5157 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:26.374565  5144 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:26.379283  5187 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:26.384013  5153 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:26.393893  4929 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:26.401399  5055 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:26.379523  5146 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:14:26.421077   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.426021  1001 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:26.431768   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.434414   999 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.396955  5188 consensus_queue.cc:1059] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:14:26.436463   751 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 10, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:26.383023  5168 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.406193  5073 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:26.367571  5070 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.403496  5137 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:14:26.445622   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.448068   997 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:26.449867   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.451268   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.409526  5005 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.416167  5144 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.408771  4763 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.464275   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 640) took 1148 ms. Trace:
I20260709 13:14:26.467774   929 rpcz_store.cc:276] 0709 13:14:25.316028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.493088 (+177060us) service_pool.cc:224] Handling call
0709 13:14:25.493400 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:25.743818 (+250418us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:25.773517 (+ 29699us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.999451 (+225934us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:26.014084 (+ 14633us) log.cc:844] Serialized 26 byte log entry
0709 13:14:26.014236 (+   152us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:26.014500 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 13:14:26.014699 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.014805 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:26.207688 (+192883us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003c1c00. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa52522 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:26.224539 (+ 16851us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b480015c258. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d07a68aa1 00007f0d07a5fb37 00007f0d07a5f9ca 00007f0d07b5726f 00007f0d07b56f37 00007f0d07b6b55c 00007f0d07b6b508 00007f0d07b6b488 00007f0d07b6b443 00007f0d07b6a6ec 00007f0d06a94a96
0709 13:14:26.464065 (+239526us) raft_consensus.cc:1685] finished
0709 13:14:26.464086 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.464219 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352119,"prepare.queue_time_us":18347,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":258866,"raft.run_cpu_time_us":4458,"raft.run_wall_time_us":243438,"spinlock_wait_cycles":21053056,"thread_start_us":250659,"threads_started":2,"wal-append.queue_time_us":221720}
I20260709 13:14:26.454149  5238 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.457383  5073 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.451215  5223 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.463676  4875 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:26.477874  5238 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:26.460259  5195 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.005s
W20260709 13:14:26.482411   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 96725.1us
I20260709 13:14:26.440770  5170 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.483461   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.423005  5196 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.482031  4875 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.522535   997 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:26.585806  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.657189   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 173786us
I20260709 13:14:26.658494   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:26.659399  5195 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:26.475795  5142 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.661780   896 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:26.679581  5160 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:26.681169   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.682076  5276 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:26.658568  4881 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.657863  4875 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.691442  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:26.692365  5153 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:26.459028  5157 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.662693  5188 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.706610   969 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:26.708240   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 287268us
W20260709 13:14:26.711226   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.711861  1000 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:14:26.712246  1000 pending_rounds.cc:85] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 5
I20260709 13:14:26.712495  1000 pending_rounds.cc:107] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:14:26.663478  5151 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:26.712724  1000 raft_consensus.cc:2889] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:26.713835   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.671482  5241 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.484391  5223 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.719650   929 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:26.719820  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.720499  1004 raft_consensus.cc:2393] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:26.718448   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:26.731537   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.737689   929 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:26.738569   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:26.738803  1002 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:26.704005  5045 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:26.742633   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:26.745592  5045 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.747684  5045 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:26.749667  4879 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:26.754743   999 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:26.757366   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:26.669684  5142 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.716889  5276 raft_consensus.cc:2764] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:26.718854  5238 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:26.737459  5027 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:26.787542  5279 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:26.789177  5084 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:26.779716  5243 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:26.790434  5243 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:26.791395  5243 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.763217  5045 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:26.794530  5170 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:26.718701  5177 consensus_queue.cc:1048] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:14:26.712916  4881 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.734527  5136 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:14:26.734936  5162 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:26.730832  5151 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.741650  4993 consensus_queue.cc:1059] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 13:14:26.820535   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 154086us
I20260709 13:14:26.760949  5245 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:14:26.824246   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:26.757998  5188 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:14:26.829069   896 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:26.831795  5174 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:26.832551  5174 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:26.834455   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:26.835300   922 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:26.842353   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 121763us
I20260709 13:14:26.796659  5194 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:26.824801  5168 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.830101  5005 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:26.853258  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 108589us
I20260709 13:14:26.857203   929 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:26.842911  5278 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:26.859455   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:26.869695   925 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:26.872524   922 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:26.860895   923 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:14:26.875957   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 89754.1us
I20260709 13:14:26.859107  1003 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
W20260709 13:14:26.866154   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 561977us
I20260709 13:14:26.860230  5167 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:26.851565  5195 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:26.882112   925 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:26.850455  5144 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.883595   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:26.888656   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.890110   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:26.891567   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.892262   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:26.892618   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:26.892874   848 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.893193   850 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.888674  5005 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:14:26.896116  5131 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:14:26.900244   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:26.901139  5131 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:26.901490  5131 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:26.902262  5131 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.904958  5176 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:26.906801  4881 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:26.908783  5148 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.002s
I20260709 13:14:26.909176  5148 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.911221  5148 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:26.884527  5278 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:26.912699  5278 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.913509  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:26.916311  5151 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.884364  5195 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:26.917623  5195 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:26.918867  5195 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:26.918053  1593 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:26.920302   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:26.921979   855 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:14:26.924054  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:26.925154  1003 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:26.926766   848 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:26.899092   821 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:26.930667   847 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:26.883210  5167 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:26.943133   997 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:14:26.944454  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:26.947973  1004 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.956162  1596 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:14:26.884169  5136 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:14:26.934573  5177 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.930569  5226 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.857005  5174 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:26.962096   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63136us
I20260709 13:14:26.883747  5282 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:14:26.976293  1593 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:14:26.960454  5223 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.938581  5151 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:26.983901   927 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:26.987195  1004 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:26.992050   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:27.008299   924 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:14:26.980086  5141 consensus_meta.cc:287] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.001s
I20260709 13:14:26.980752  4780 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:27.012001   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:26.967497  5146 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:26.985212  5137 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:27.013990  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:27.021276  1004 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:27.029342   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.016819  5142 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:26.985050  5177 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.023728  5137 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:27.040637   895 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:27.025056  4780 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:27.020535  5141 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:26.996855  5226 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.060397   997 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:27.059856   926 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:27.012426  5151 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.059268   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2160) took 1184 ms. Trace:
I20260709 13:14:27.065809  1000 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:27.061808   921 rpcz_store.cc:276] 0709 13:14:25.869735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.875686 (+  5951us) service_pool.cc:224] Handling call
0709 13:14:27.054016 (+1178330us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400446bd4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:27.054136 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837087616}
I20260709 13:14:27.024654  5146 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.068399   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.068604  5141 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:27.069928  5151 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.070338  1597 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:27.080205  5293 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:14:27.068399  4929 raft_consensus.cc:3055] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:27.081084  4929 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:27.041652  5144 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.081900  4929 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.082268  5144 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.084406  5144 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.085122  5177 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:27.085285  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:27.045189  5294 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:14:27.010206  5167 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:27.070897  5289 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:14:27.086156  1000 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:14:27.074431   848 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
W20260709 13:14:27.087271   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.113003   821 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:27.088023   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:27.088634  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:27.090909   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:27.120719   969 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:27.113883   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:27.113884   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:27.123705   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 88546.3us
I20260709 13:14:27.112008  5167 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:27.158833   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.086618  5295 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:27.112166  5151 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.114324  4763 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.173316   853 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:27.124048  5136 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:27.127544  5279 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.127889  5276 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:27.184352   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70543.7us
I20260709 13:14:27.090776  5142 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:14:27.146471  5241 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:27.190071   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.206506   853 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:27.202320   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:27.130342  5287 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:27.252003   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.129637  4881 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:14:27.180540  5168 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.342710  5276 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:27.212495  5241 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:27.204062  5286 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.340763  5287 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.175005  5151 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.366091   853 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:27.181682  4763 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.344846  4879 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.129066  5137 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:27.372457  5241 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:27.392462  5311 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:27.393168  5311 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.393579  5311 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.172029  4780 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.202018  5279 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.373859   850 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:27.360350  5136 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:27.376646  5287 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:27.404866  5162 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:27.462235  5115 consensus_meta.cc:287] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 1.276s	user 0.015s	sys 0.016s
I20260709 13:14:27.472149  5223 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:27.472052   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 358206us
I20260709 13:14:27.364277  5168 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.477182  5170 consensus_peers.cc:313] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Unable to generate Tablet Copy request for peer: Not found: Peer not tracked or queue not in leader mode.
W20260709 13:14:27.488399  5308 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:14:27.489236  5302 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:27.489744  5306 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:14:27.492571  5084 tablet_bootstrap.cc:700] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Time spent opening tablet: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:14:27.500454   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.501175   853 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:27.502264  5144 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:27.481559  5176 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:14:27.454357  5280 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:27.406589  4879 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.488473  5305 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.376196  5286 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.457010  4940 consensus_meta.cc:287] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.047s	user 0.017s	sys 0.000s
I20260709 13:14:27.524346   850 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
W20260709 13:14:27.553027  5296 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:14:27.527876  1596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 452) took 1176 ms. Trace:
I20260709 13:14:27.532810   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.530040  5177 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:27.552726   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.493042  5295 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.582130  1596 rpcz_store.cc:276] 0709 13:14:26.351392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.351518 (+   126us) service_pool.cc:224] Handling call
0709 13:14:26.363164 (+ 11646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:26.961245 (+598081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.527508 (+566263us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54001e37d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:27.527798 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362765184}
I20260709 13:14:27.554627   751 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 8, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 8 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:27.003525 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:27.558627  1004 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:27.566929  4879 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:27.567368  5300 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:14:27.503816  5055 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 1.062s	user 0.010s	sys 0.003s
W20260709 13:14:27.580849  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94611.3us
I20260709 13:14:27.486575  5223 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.671900  5223 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.573292  5305 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.678103  5305 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:27.561431  5144 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.679700   955 heartbeater.cc:646] Failed to heartbeat to 127.31.120.62:45937 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.31.120.62:45937 timed out after 15.000s (SENT)
I20260709 13:14:27.680042  5144 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.562587  5280 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.524197  5035 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:27.688865  5035 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.689568  5280 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.564729  5146 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:27.596984   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.607446   852 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:27.738708   967 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.003s
W20260709 13:14:27.740660   967 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.006s	sys 0.003s
W20260709 13:14:27.741818  4875 consensus_meta.cc:287] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.002s
W20260709 13:14:27.739457   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 213119us
I20260709 13:14:27.536450  5302 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:27.751868   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 533375us
I20260709 13:14:27.657027  5177 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.759068  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:27.761339  1001 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:27.751374  5055 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.767489  5055 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.768249  5196 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:27.769662  5055 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:27.773689   751 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 10, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:27.654316  5295 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.555887  5276 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.775305   751 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 10, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:27.777850  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:27.778745  5309 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:14:27.761498  5136 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:27.785502   847 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:14:27.810546   847 pending_rounds.cc:85] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
I20260709 13:14:27.810865   847 pending_rounds.cc:107] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:14:27.811138   847 raft_consensus.cc:2889] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.786330  5226 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.760210  4875 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.801962   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45723.8us
I20260709 13:14:27.812769   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:27.814039  4875 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.814831   855 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:27.762904  5035 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.791316  5151 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:27.819774  5304 log.cc:927] Time spent T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.454s	user 0.002s	sys 0.005s
I20260709 13:14:27.836347  5168 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.802788  4763 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.844920  5302 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:27.849551  5295 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.802125  5176 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.835778  5279 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.823350  5305 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:27.858390   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45743.1us
I20260709 13:14:27.897027   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:27.850996  5287 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.900081   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:27.902486   924 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:27.904229   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:27.829939  5144 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:27.862262  5195 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.941s	user 0.014s	sys 0.000s
I20260709 13:14:27.863005  5035 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:27.820232  5136 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.914031   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 103121us
I20260709 13:14:27.917017  1002 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:14:27.920115  1002 pending_rounds.cc:85] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 4
I20260709 13:14:27.920432  1002 pending_rounds.cc:107] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:14:27.920732  1002 raft_consensus.cc:2889] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.921368   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:27.922554   924 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:27.901271  5177 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:27.864853  5320 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.913048  5179 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:27.904330  5321 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.861615  5226 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:27.939134   924 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:27.918582  5035 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:27.857892  5317 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:14:27.905332  5305 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.936519  5241 consensus_meta.cc:287] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.002s
I20260709 13:14:27.913826  5176 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.943696  5035 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:27.951905  5119 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:27.952545  5119 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.954703  5119 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.956266  5156 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:27.956682  5156 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:27.957001  5136 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:27.957526  5156 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:27.958998  5311 consensus_meta.cc:287] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:14:27.959420  5311 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.943800  5321 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.960309  5305 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.960541  1003 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:27.940372  5320 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.961781  5321 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.963624  5320 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.963725  5241 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.965063  5287 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:27.966240  5241 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:27.967037  5311 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:27.966882  5223 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.968447  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:27.921671  5144 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.969161   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 2057) took 1071 ms. Trace:
I20260709 13:14:27.969569   855 rpcz_store.cc:276] 0709 13:14:26.898228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.934182 (+ 35954us) service_pool.cc:224] Handling call
0709 13:14:26.934781 (+   599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.815227 (+880446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.968921 (+153694us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b54000ffd54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:27.969105 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2023316224}
I20260709 13:14:27.971231  5144 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:27.974246  5321 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:14:27.974730   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.976649  5136 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:27.980815  4929 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:27.984784  5238 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:27.985376  5238 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:27.986518  5238 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:27.997663   849 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:27.969182  5176 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:27.988637  5326 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:14:27.991019  5223 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:27.992782  5303 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.422s	user 0.004s	sys 0.004s
I20260709 13:14:27.992686   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.993065  5035 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:27.944898  5226 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.013271   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:27.993952  5315 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.941506  5179 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.999863   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.007414   847 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:28.008677   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.022842   854 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:28.012117   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.011838  1002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 2128) took 1126 ms. Trace:
I20260709 13:14:28.005635  5311 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.024706   751 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 11, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:28.025820  5311 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.027985  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.028250  5311 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.006533  5170 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.028883  5226 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.016492   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2173) took 1023 ms. Trace:
I20260709 13:14:28.029295  5170 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.029958   928 rpcz_store.cc:276] 0709 13:14:26.992988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.993751 (+   763us) service_pool.cc:224] Handling call
0709 13:14:26.994121 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.012660 (+ 18539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.012699 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.013266 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:14:27.200170 (+186904us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.200209 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.200286 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:27.992258 (+791972us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:28.016216 (+ 23958us) raft_consensus.cc:1685] finished
0709 13:14:28.016239 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.016437 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18164,"raft.queue_time_us":27971,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":8863,"spinlock_wait_cycles":6345088,"thread_start_us":165991,"threads_started":1,"wal-append.queue_time_us":455055}
I20260709 13:14:28.033929  5170 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.032428  1593 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:14:28.034624  1593 pending_rounds.cc:85] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 8
I20260709 13:14:28.034878  1593 pending_rounds.cc:107] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:14:28.035126  1593 raft_consensus.cc:2889] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.036505  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.024362  1002 rpcz_store.cc:276] 0709 13:14:26.885701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.887208 (+  1507us) service_pool.cc:224] Handling call
0709 13:14:26.900083 (+ 12875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.921326 (+1021243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.009641 (+ 88315us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400359654. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:28.011748 (+  2107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337573632}
I20260709 13:14:28.037237  1593 raft_consensus.cc:2393] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:28.037420  5320 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:28.032377   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.049021   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:28.049754   999 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.046372   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:28.051683   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:28.014046   846 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:28.053190  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.054333   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.027500   751 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 11, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.062707  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:28.068132   922 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:28.079348  5115 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.079977  5115 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.081993  5115 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:28.082775   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.083668  5195 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:28.084077  5195 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.084220   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.084990   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.085781  5302 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:14:28.086407   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.087869  5144 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:14:28.089176  5167 consensus_meta.cc:287] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.916s	user 0.008s	sys 0.002s
I20260709 13:14:28.089959   852 raft_consensus.cc:2410] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:14:28.087319   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.090946  5144 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:28.091209   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.091355  5144 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:28.091853   853 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 4.
I20260709 13:14:28.092221  5144 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.092686   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.093087   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.093691   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.094513   853 raft_consensus.cc:2410] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:14:28.038048  5179 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:28.065346  5277 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.096243  5277 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.098511  5277 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.065851  5241 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.093379  5144 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.099460  5241 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.101697  5241 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:28.093724   847 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:28.076435  5311 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:14:28.066239  5223 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:14:28.104206   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.076864  5136 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:28.104848   851 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:28.105193  5136 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.105566  5136 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.105784   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.106346   851 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:28.108026  5035 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:28.109668  5284 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.100s	user 0.007s	sys 0.014s
I20260709 13:14:28.113314   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2174) took 1118 ms. Trace:
I20260709 13:14:28.078014   999 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:28.084463  5195 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:28.115502   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50615.3us
I20260709 13:14:28.115583   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.084950   854 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:28.113745   927 rpcz_store.cc:276] 0709 13:14:26.994639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.005704 (+ 11065us) service_pool.cc:224] Handling call
0709 13:14:27.006021 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.470972 (+464951us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:27.471335 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 13:14:27.749817 (+278482us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.750368 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:14:27.750503 (+   135us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.750539 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.112945 (+362406us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540031b5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:28.112991 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.113032 (+    41us) raft_consensus.cc:1685] finished
0709 13:14:28.113127 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.113242 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":358336,"prepare.queue_time_us":12879,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":359350,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":1469,"spinlock_wait_cycles":244127616,"thread_start_us":277984,"threads_started":1,"wal-append.queue_time_us":307420}
I20260709 13:14:28.119225   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.119800   849 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.120609   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:28.121451   927 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:28.130108   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:28.132205   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.135766   819 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:28.137141  5279 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:28.137655  5279 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.049150  5315 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.142019  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.143298   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:28.145849  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:14:28.147871   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.149133   927 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:28.153370   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.139919  5241 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:28.159893   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.162604   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.163673   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:28.170482  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.154458   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:28.139508  5279 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:28.174278  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.174629   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.187788   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.176087   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 2172) took 1310 ms. Trace:
I20260709 13:14:28.188482   925 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:28.186316  1601 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 6.
I20260709 13:14:28.184392   853 raft_consensus.cc:2410] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:14:28.188874   926 rpcz_store.cc:276] 0709 13:14:26.865203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.865313 (+   110us) service_pool.cc:224] Handling call
0709 13:14:27.054560 (+189247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.061293 (+  6733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.175829 (+1114536us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400446bd0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:28.176018 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434574592}
W20260709 13:14:28.197492   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.199085   854 raft_consensus.cc:1240] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:28.202214   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47866.3us
I20260709 13:14:28.195574  5321 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.208017  5321 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.211062   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2171) took 1220 ms. Trace:
I20260709 13:14:28.211534   930 rpcz_store.cc:276] 0709 13:14:26.990275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.994692 (+  4417us) service_pool.cc:224] Handling call
0709 13:14:26.995160 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:26.995483 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:26.995506 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.969004 (+973498us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b2c0000e5c8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:14:27.969580 (+   576us) log.cc:844] Serialized 499 byte log entry
0709 13:14:28.145215 (+175635us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:28.145266 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.145356 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.210517 (+ 65161us) raft_consensus.cc:1685] finished
0709 13:14:28.210540 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.210951 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5495552,"thread_start_us":175330,"threads_started":1,"wal-append.queue_time_us":239352}
I20260709 13:14:28.156353  4940 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:28.223744   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:28.225512   853 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:14:28.226040   853 pending_rounds.cc:85] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 7
I20260709 13:14:28.225970  1004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 564) took 1185 ms. Trace:
I20260709 13:14:28.226596  1004 rpcz_store.cc:276] 0709 13:14:27.040803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.043396 (+  2593us) service_pool.cc:224] Handling call
0709 13:14:27.043829 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.606176 (+562347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.225746 (+619570us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54000898d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:28.225917 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182797696}
I20260709 13:14:28.228368   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:28.238143   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:28.186667  5241 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:28.197196  5276 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:28.226505   853 pending_rounds.cc:107] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:14:28.210345  5321 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:28.171108  5115 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:14:28.222855  5279 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:28.249863   853 raft_consensus.cc:2889] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.248558   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.259748   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.246342  5196 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:28.236814  5162 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:28.262565  5162 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.263104  5162 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.227646  4780 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:14:28.237064  4875 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:14:28.268895  5179 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.269701  5286 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:28.270982  5179 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.271973  1001 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:28.273312  5136 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.275252  5136 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.248483  5160 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:28.276166  5160 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.276566  5160 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.231870  5316 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:28.278841  5316 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.280952  5316 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.281078   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.259086  5279 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:28.281729  5279 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.288002   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:28.298476  5341 log.cc:927] Time spent T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:14:28.340476  5005 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.356943   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.357318  5005 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.360728  5005 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.361205  4780 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:28.362530  5137 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:28.340052  5338 log.cc:927] Time spent T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.060s	user 0.002s	sys 0.004s
I20260709 13:14:28.365906  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:28.282743  5328 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.339793  5196 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:28.339246  5195 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.295168  5155 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.351539   849 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
W20260709 13:14:28.339012  5340 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.062s	user 0.001s	sys 0.003s
I20260709 13:14:28.356065   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:28.384390   930 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:28.314035  5241 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.340160  5269 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:28.366245   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.366271   819 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:28.400127   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.392840  5276 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:28.406507  1004 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:28.409345   819 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:28.409624  1004 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:28.412109  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.414943  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:28.416374   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 73045.4us
I20260709 13:14:28.417807   930 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:28.418251  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.420915   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.421471  1004 raft_consensus.cc:2410] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:28.394399  5269 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:28.403589  5311 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:28.403867  5134 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.411439  4780 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.492012  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.382965  5115 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:14:28.492726  1004 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:28.394611  5328 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.420442  5267 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.496479  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.497216  1001 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:28.401826  5155 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.500294   927 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:28.501284   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.491109  5322 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:28.474375  5113 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:28.503335  5113 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.503808  5113 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:28.504021  5344 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:14:28.505396   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.506677   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.502660   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.508582   921 raft_consensus.cc:2393] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:28.509775   921 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:28.454205  5295 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.502553   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.516832  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:28.520936  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.520936   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:28.528266   921 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:28.531520   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.519447   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:28.526665   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.534479   853 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.537746   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:28.538965   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.540226   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.540845   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.513340  5352 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.540902   921 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:28.560920   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.540275   927 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:28.543027   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:28.544180   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.569166  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:28.536492  5005 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:28.543788  5214 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:28.584182  5214 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.584101  5005 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.584693  5214 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:28.586508  5005 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:28.588949   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45977.3us
I20260709 13:14:28.590265   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:28.591928  5226 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:28.592303  5226 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.592707  5226 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.595649   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.596385   852 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:28.497865  5311 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.598023   850 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:28.499961  5134 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.522218  5286 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.520970  5136 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.619028   853 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:28.545194  5321 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:28.630628   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.631812   850 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:28.643853   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:28.540866  5245 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:14:28.598445  5035 raft_consensus.cc:2764] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:28.605644  5320 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:28.662314  1003 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:28.663501   848 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:14:28.664039   848 pending_rounds.cc:85] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 5
I20260709 13:14:28.664311   848 pending_rounds.cc:107] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:28.664587   848 raft_consensus.cc:2889] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.666345   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.653664   852 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:28.653961   855 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:28.517930  5267 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.677215  5267 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:28.677342  5351 log.cc:927] Time spent T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:14:28.678871  5342 log.cc:927] Time spent T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:14:28.627014  5286 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.682164   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2169) took 1709 ms. Trace:
I20260709 13:14:28.682528   929 rpcz_store.cc:276] 0709 13:14:26.973166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.974204 (+  1038us) service_pool.cc:224] Handling call
0709 13:14:26.974651 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:26.975113 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:26.975154 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.993847 (+1018693us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b2c0000e5c8. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:14:27.994237 (+   390us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.237994 (+243757us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:28.238039 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.238112 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.681883 (+443771us) raft_consensus.cc:1685] finished
0709 13:14:28.681905 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.682104 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14200,"raft.queue_time_us":37484,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":1955,"spinlock_wait_cycles":13968256,"thread_start_us":243087,"threads_started":1,"wal-append.queue_time_us":430435}
I20260709 13:14:28.524577  5295 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.670538  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.684280  5348 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.684962  1002 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:28.684892  5348 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.625543  5136 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.608728  5302 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.686139  5302 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.687129  5136 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.535591  5322 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.692392  1002 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:28.687076  5348 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.694382  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.695319  5311 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.697642  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.698501  1000 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:28.699962  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.700071   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.700982   998 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:28.595188  5214 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.702039  5321 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:28.703239  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.704216  5134 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.705332   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.705961  1000 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:28.706400   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.707114   852 raft_consensus.cc:2393] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:28.708223  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.709053  1002 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:28.710661  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:28.711448  1000 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.712450  5328 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.713687  5195 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:28.704252  1003 raft_consensus.cc:2368] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:28.608144  5347 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:28.529126  5196 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.639138  5315 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.719377   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.732712   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:28.733299  1000 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
W20260709 13:14:28.741755   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.741946   998 raft_consensus.cc:2410] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:14:28.741811   852 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:28.742693  4929 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.743456  4929 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:28.743120   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:28.746167  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:28.739521  5084 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.064s	user 0.002s	sys 0.002s
I20260709 13:14:28.751930   855 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:28.753194   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:28.753933   995 raft_consensus.cc:2368] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:28.575456  5352 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.757494   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:28.768657   928 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:28.769058   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.599402  5276 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.758530   895 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:28.769685  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:28.757543   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.185213  1000 raft_consensus.cc:2393] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:28.880497   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.747051  5195 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.188045  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:28.927347  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 358225us
I20260709 13:14:29.188292  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.195060  1003 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:29.188066   967 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:29.192701   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 434229us
W20260709 13:14:28.998258  5084 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:29.199208   895 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
W20260709 13:14:29.028625  5362 log.cc:927] Time spent T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:14:29.199678   895 leader_election.cc:403] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:14:29.209430   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.758100  5113 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.212527  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.214169  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:28.866619  5070 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.214803  1002 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.214872  5070 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:28.796669  5315 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.216084  5315 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.217214  5070 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.217794  5070 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.219172  5315 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.842581  5320 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.747293  5160 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.226459  5320 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.227901  5160 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.228958  5160 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.228996  5320 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.751078  5148 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.229485  5160 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.229957  5148 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.219897  5134 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:28.841898  4929 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:28.740356  5162 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:28.747166  5279 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.199654  5195 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:29.279145  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.281667  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:29.282135  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.282770  1599 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:28.842144  5364 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.232406  5148 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.231554  5070 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:29.292146  1002 raft_consensus.cc:2468] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:29.283397  5134 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.263024  5187 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:29.232133  5378 consensus_queue.cc:1059] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:14:29.309705   997 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:28.746475  5347 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.261232  5174 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:29.079586  5226 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.231540  5160 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.310945  5187 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:29.320129  5084 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20260709 13:14:29.323736  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 127104us
W20260709 13:14:29.325130   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 127081us
I20260709 13:14:29.294620  5364 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.326864  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.327960  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.328217  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.328934  1002 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:29.329964  1002 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:14:29.330976   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.331574   926 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:29.332525   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.332813   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.333611   926 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:29.335906   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.336786   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:29.335974  5328 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.338138  1002 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:29.337885  5328 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.340425   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:29.341192   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.344213   928 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 14: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:29.347656   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2180) took 1452 ms. Trace:
I20260709 13:14:29.348019   922 rpcz_store.cc:276] 0709 13:14:27.896522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.896692 (+   170us) service_pool.cc:224] Handling call
0709 13:14:29.347422 (+1450730us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003cbc54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:29.347531 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391216000}
I20260709 13:14:29.330997  4929 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.350946  5134 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.354066  5084 tablet_bootstrap.cc:492] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:29.356177  5195 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.346654  5177 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.357332  5162 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.357726  5177 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.358826  5279 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.358953  5328 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.359889  5177 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.364275  5084 ts_tablet_manager.cc:1403] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 2.608s	user 0.043s	sys 0.029s
I20260709 13:14:29.364532  5113 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.365840  5347 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.354233  4929 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.377624  4929 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.380396  5226 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.381399  5364 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.390194  5322 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.366050  5187 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.416752  5238 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:29.439558   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 102432us
I20260709 13:14:29.440659   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.443810   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.446621   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.120047 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:29.448189   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.450721   751 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 9 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.452198   751 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 1 to 7, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.444989  5285 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:29.453568  5170 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.462833  5285 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.463186  5170 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.463395  5285 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:29.463719  5170 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.453820  4940 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:29.467273  4940 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.467701  4940 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.469328   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 2084) took 1379 ms. Trace:
I20260709 13:14:29.469722   847 rpcz_store.cc:276] 0709 13:14:28.103000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.105610 (+  2610us) service_pool.cc:224] Handling call
0709 13:14:28.118564 (+ 12954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.119096 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.119150 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:28.125102 (+  5952us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.807999 (+682897us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b18004a0080. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df0d1 00007f0d02afaa0f
0709 13:14:29.147927 (+339928us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:29.147985 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.148084 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:29.225232 (+ 77148us) raft_consensus.cc:1685] finished
0709 13:14:29.225260 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.469151 (+243891us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b54003b06d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3e8ad 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:29.469271 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5217,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":12040,"spinlock_wait_cycles":35128960,"thread_start_us":309846,"threads_started":1,"wal-append.queue_time_us":1098437}
I20260709 13:14:29.478042  5388 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.478667  5388 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.450145  5276 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.479981  5055 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:29.482232  5369 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.194s	user 0.003s	sys 0.000s
I20260709 13:14:29.481756  5055 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:29.484107  5055 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.477317  5347 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.485195  5347 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.487309  5347 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.488543  5302 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:29.489455  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 196366us
I20260709 13:14:29.496387  5131 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.497452  5352 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.497663  5382 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.498152  5382 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:29.499459  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 174676us
I20260709 13:14:29.498896  5295 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.507995  5382 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.509486  5214 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.479563  5276 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.518810  1598 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:29.491827  5388 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.519383  5374 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.521473  5374 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.522336  5168 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:29.522857  5168 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:29.525709  5374 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.526362  5383 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:14:29.526813  5170 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.528117   846 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:29.530354  5269 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:29.536266   846 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 674) took 1310 ms (client timeout 1606 ms). Trace:
I20260709 13:14:29.536813  5084 raft_consensus.cc:359] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.538437  5084 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:29.540946  5156 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:29.543211  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51515.9us
I20260709 13:14:29.545303   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.545469  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:29.546295  1003 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:29.546912  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.547806   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.548426   997 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:29.549348  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.549751  5155 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.549932  1001 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:29.552224   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 131872us
I20260709 13:14:29.552812   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.518747  5276 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:29.536702   846 rpcz_store.cc:269] 0709 13:14:28.225515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.227323 (+  1808us) service_pool.cc:224] Handling call
0709 13:14:29.536013 (+1308690us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54002b9154. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:29.536133 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2916201216}
I20260709 13:14:29.553733  1001 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:29.556907  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.555197   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.553462   997 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:29.555825  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.531519  5269 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.523701  5168 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.563390  5168 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.559724  5187 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.565373  5187 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.562001  5280 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
W20260709 13:14:29.567813  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67747.3us
I20260709 13:14:29.558550  5316 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.569732  5316 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.570526   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:29.560634  5388 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:29.571746  5316 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.572085  5388 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.574759  5388 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.571589  5286 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.575938   751 catalog_manager.cc:5697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 2 to 11. New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.539147  5084 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:29.578442   751 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 9, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.579984   751 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.581458   751 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 9 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.582264   751 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 10, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.552608  5285 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.567478  5187 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:29.592784   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.605870   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.595791   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.618782   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:29.585650   750 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 232) took 17425 ms (client timeout 15000 ms). Trace:
I20260709 13:14:29.595839   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.587890   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2194) took 1230 ms. Trace:
I20260709 13:14:29.597784   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:29.597368   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.623791   922 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:29.584033   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:29.609405  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.627954   998 raft_consensus.cc:606] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:29.628494  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44664us
I20260709 13:14:29.629351  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.625217   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.630154   930 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:29.630381  5170 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.632437  5285 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.586440  5084 ts_tablet_manager.cc:1434] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.222s	user 0.008s	sys 0.000s
I20260709 13:14:29.633898  5378 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:29.584426  4940 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.636739  1004 raft_consensus.cc:2393] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:29.613093   751 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 9 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.638747   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.639905   751 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.641106   751 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.623811   925 rpcz_store.cc:276] 0709 13:14:28.357263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.357466 (+   203us) service_pool.cc:224] Handling call
0709 13:14:28.357715 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.357911 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:28.999704 (+641793us) log.cc:844] Serialized 14 byte log entry
0709 13:14:29.587527 (+587823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:29.587581 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:29.587664 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.587738 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.587837 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":843136,"prepare.queue_time_us":564922,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":390148,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":384452,"threads_started":2,"wal-append.queue_time_us":276513}
I20260709 13:14:29.639362   927 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:29.646250   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.646786   927 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:29.646133   751 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 11, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.650097  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:29.650970  1593 raft_consensus.cc:2393] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 6: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:29.651623  5360 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.651934   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.652164  5360 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.653488  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.653035   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 7cf198f3795b43ff951a11167bc21580 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:42376
I20260709 13:14:29.656598  5378 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:29.657040  5378 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:29.657442  5378 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.659271   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.660845   925 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:29.645284  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.647892   751 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:29.665504   751 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.618652  5328 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.624125  5302 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.591979  5174 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:29.657963  5360 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:29.583151  5131 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:14:29.668322   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.745389   999 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:29.766799   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:29.685503   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46867us
I20260709 13:14:29.667853   751 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 6, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 6 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:29.773811  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 128602us
I20260709 13:14:29.774850  1567 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:29.773778   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.764753   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 96477.8us
I20260709 13:14:29.668751   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.774586   751 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 5 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.811107  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.779155   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:29.811847   929 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:29.811908   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.645475  5286 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:29.812686   751 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:29.815274  1598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37790 (request call id 719) took 1824 ms (client timeout 1567 ms). Trace:
I20260709 13:14:29.815171   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:29.812103   896 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.854s (SENT) [suppressed 7 similar messages]
I20260709 13:14:29.817353   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:29.817937   924 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.711984   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-cdfea09a06ff4100a0b80619f35f7008 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:14:29.818701   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-cdfea09a06ff4100a0b80619f35f7008 on tablet cdfea09a06ff4100a0b80619f35f7008 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:29.818928  5316 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:29.816475  5399 raft_consensus.cc:2804] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:29.821255  5399 raft_consensus.cc:697] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:29.822844   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:29.815913  1598 rpcz_store.cc:269] 0709 13:14:27.968034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.968183 (+   149us) service_pool.cc:224] Handling call
0709 13:14:29.792025 (+1823842us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540024ead4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:29.792191 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3563911040}
I20260709 13:14:29.817816   751 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 7, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 7 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:29.823592   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 226710us
I20260709 13:14:29.824424   751 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:29.825536   751 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 3 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:29.686432  4993 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.722592  5141 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:29.760329   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 7cf198f3795b43ff951a11167bc21580 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
W20260709 13:14:29.844807   737 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 177751us
I20260709 13:14:29.845409   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:29.845587   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:29.846145   930 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.832233  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:29.847848  1599 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:29.825311   821 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:29.764864  4929 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.840236   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.848706   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.850389   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:29.707615  5396 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:14:29.852913   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.882622   896 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.883014  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.887837  4875 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.888406  4875 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.889160  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.813706  5315 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.785091  4940 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.814429  5223 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:29.891855   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.853611   969 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.893940   927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:14:29.899960   927 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:29.900717   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 75625.8us
I20260709 13:14:29.841339  5280 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.840411  5176 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:29.905097   896 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:29.809849  5328 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.893334  5384 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:29.921139   924 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:29.907464   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:29.922230   848 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.908375   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:29.923206  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 192646us
I20260709 13:14:29.923647   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:29.923218   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.924821   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:29.925387  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.908571   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.925801   818 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.926004  1598 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:29.902666  5214 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:29.912148  5151 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:29.822225  5399 consensus_queue.cc:237] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.912566   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:29.913352   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59833us
I20260709 13:14:29.931305   854 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
W20260709 13:14:29.905956   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55641.7us
I20260709 13:14:29.910301  5142 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:29.894708  5377 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.880944  5144 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:14:29.881069  5363 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:29.893131  4929 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.917352   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.907510   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.915009  5347 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:29.920043  1596 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:29.882098  5168 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:29.926647   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.898303  5223 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.827844  5286 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.823174  5302 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.898484  5269 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.904629  5241 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.905164  5155 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.934010  5267 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.957525  5361 raft_consensus.cc:3055] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:29.957975  5361 raft_consensus.cc:740] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:29.957695  5267 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.959517  5302 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.959729  5238 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:29.960067   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.960321  5269 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.961527   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.948136  5113 consensus_queue.cc:1059] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:14:29.962602  5279 raft_consensus.cc:3055] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:29.928336  5176 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:29.910748  5115 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:29.963454  5115 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.956759  5196 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:29.963900  5115 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:29.964136  5196 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.934989  5243 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:29.967453  5176 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.966672  5243 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.952417  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.958905  5361 consensus_queue.cc:260] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:29.916219  4940 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:29.933043   967 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:29.934227   999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 2098) took 1436 ms. Trace:
I20260709 13:14:29.881573  4993 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.975358  4940 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:29.980155  4940 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.983475  5365 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.983726  5156 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:29.983531   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50560us
I20260709 13:14:29.937463   930 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
W20260709 13:14:29.623034   750 rpcz_store.cc:269] 0709 13:14:12.160149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.160409 (+   260us) service_pool.cc:224] Handling call
0709 13:14:29.585459 (+17425050us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:22.073650 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:22.074395 (+   745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:22.074451 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:14:22.074482 (+    31us) tablet.cc:662] Decoding operations
0709 13:14:22.098473 (+ 23991us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:22.130286 (+ 31813us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:22.131972 (+  1686us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:14:22.330683 (+198711us) tablet.cc:701] Row locks acquired
0709 13:14:22.330746 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:14:22.331094 (+   348us) write_op.cc:270] Start()
0709 13:14:22.331309 (+   215us) write_op.cc:276] Timestamp: P: 1783602862330942 usec, L: 0
0709 13:14:22.331364 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:14:22.332321 (+   957us) log.cc:844] Serialized 3990 byte log entry
0709 13:14:22.743436 (+411115us) op_driver.cc:464] REPLICATION: finished
0709 13:14:22.868685 (+125249us) write_op.cc:301] APPLY: starting
0709 13:14:22.906719 (+ 38034us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:23.016419 (+109700us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:23.016576 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:23.362813 (+346237us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:23.363167 (+   354us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=20
0709 13:14:23.363219 (+    52us) write_op.cc:312] APPLY: finished
0709 13:14:23.363658 (+   439us) log.cc:844] Serialized 139 byte log entry
0709 13:14:23.391802 (+ 28144us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:23.445818 (+ 54016us) write_op.cc:454] Released schema lock
0709 13:14:23.472284 (+ 26466us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:23.532014 (+ 59730us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:23.533175 (+  1161us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76049,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":88549,"apply.run_cpu_time_us":19484,"apply.run_wall_time_us":663420,"cfile_cache_hit":18,"cfile_cache_hit_bytes":18810,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":1,"lbm_read_time_us":73594,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":6,"num_ops":10,"prepare.queue_time_us":213692,"prepare.run_cpu_time_us":16321,"prepare.run_wall_time_us":328765,"raft.queue_time_us":176062,"raft.run_cpu_time_us":10401,"raft.run_wall_time_us":67288,"replication_time_us":411883,"thread_start_us":238745,"threads_started":4,"wal-append.queue_time_us":215085,"wal-append.run_cpu_time_us":55885,"wal-append.run_wall_time_us":1927359}]]}
I20260709 13:14:29.916989  5317 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.939321  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.939635   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:29.930125  5214 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:29.924160  5275 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:14:29.924955  5384 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.945861  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.924701  5188 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:29.945956   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.918000  4875 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:29.947978   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:29.962957  5279 raft_consensus.cc:740] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:29.972923  1596 raft_consensus.cc:2410] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:14:30.047252   849 raft_consensus.cc:2368] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 6. Current term is 7.
I20260709 13:14:29.972719  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:29.921617  5174 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:29.977618  5401 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:14:29.975533   999 rpcz_store.cc:276] 0709 13:14:28.493586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.501028 (+  7442us) service_pool.cc:224] Handling call
0709 13:14:29.929308 (+1428280us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400359b54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:29.929413 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859350912}
I20260709 13:14:29.986065   967 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:29.990770   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 64957.9us
W20260709 13:14:29.994360   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54814.4us
I20260709 13:14:29.945773  5354 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:30.093799   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.995129   848 raft_consensus.cc:2410] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:29.946597  5295 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:29.957440  5241 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.956167  5168 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.984110  5156 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.972060  5347 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.054661  5384 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:30.008124  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.046332  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.048117  1599 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:29.932946   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.119632   855 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:29.992707  5113 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.976046  5155 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.054028   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.124650   849 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:30.125798   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.127195   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:30.066468   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.075874   818 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:29.938195  5363 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:30.161085  5363 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.098348   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:30.161355   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85592.6us
W20260709 13:14:30.162163   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63923.9us
I20260709 13:14:30.163007   896 leader_election.cc:400] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:30.163753  5113 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:30.163964  1566 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:30.164362  5113 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.164834  5302 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:30.165171  5277 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:30.165382  5302 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.165930  5302 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.995975  5214 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:30.099797   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 113784us
I20260709 13:14:29.984067  5365 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:29.997256  5226 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:30.116448   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.117197  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:30.170902   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:30.171448  1596 raft_consensus.cc:2393] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:30.094945  5188 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:30.121521   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:29.972221  5035 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:14:30.100835  5361 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.128829  5286 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.128988  5155 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.109477  5156 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:30.109102  5170 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:30.130874   847 raft_consensus.cc:2410] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:14:30.131642   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.189139  5170 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:30.189666  5170 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:30.190191   855 raft_consensus.cc:2410] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:30.191884   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.193399   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.194391   855 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:29.974490  5223 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:30.197821  5055 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.198025   969 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.198360  5055 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.198894  5315 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:30.200476  5055 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.104983  5153 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.148988  5328 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.201954  5153 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.209575  5354 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:30.210284  5354 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.213022  5354 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.149514  5168 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.151481  5378 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.215451  5378 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.153692  5115 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:30.216431   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.217813  5115 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.221341  5188 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.222831   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.145156  5279 consensus_queue.cc:260] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.225059  5279 raft_consensus.cc:3060] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:30.225628   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.164454   896 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:30.168887   849 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:30.234081  5363 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.167797  5131 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:30.235975  5131 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:30.236425  5131 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.237795   849 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:30.181568  5388 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:30.238785  5388 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:30.238873   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.171995   967 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:30.172859  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.175638   854 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:30.239790  5388 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.245594  5131 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.165947  5277 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:14:30.110514  5320 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:14:30.128830  5241 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.184738   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.249151  5347 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.250394   848 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:30.250478  5347 raft_consensus.cc:2804] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:30.253341   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 88973.2us
I20260709 13:14:30.254168   896 leader_election.cc:400] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:30.254616  5364 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:30.234658  5365 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:29.995826  5317 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.180832  5285 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:30.169234   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:30.195473  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.277983  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.279418  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.280396  1599 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:30.235306   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:30.240039   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 68236.3us
I20260709 13:14:30.287927   855 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:30.292955   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.294170   924 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:30.295426   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.254979   896 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:30.255559  5378 consensus_queue.cc:1059] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:30.258901  5277 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:30.262506  5276 consensus_meta.cc:287] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.000s
I20260709 13:14:30.314219   849 raft_consensus.cc:2410] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:14:30.229549  5223 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.168486  5113 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.251276  5347 raft_consensus.cc:697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:30.321920   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:30.326642   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 71726.7us
W20260709 13:14:30.328816   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 165174us
I20260709 13:14:30.333803   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.324828  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:30.340732  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.339534   928 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:30.347285   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.351938   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.352108   818 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.330684  5142 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:30.364694   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:30.361215  5115 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:30.369369  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:30.370112  5226 consensus_queue.cc:1059] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:30.370879  5226 consensus_queue.cc:1243] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:30.332273  5155 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:30.374752   896 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:30.349793  5141 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:30.379056  5168 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.383303   849 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:14:30.384503   849 pending_rounds.cc:85] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
I20260709 13:14:30.384801   849 pending_rounds.cc:107] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:14:30.385074   849 raft_consensus.cc:2889] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.351173  5241 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:30.326257  5285 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.327553  5276 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.332351  5277 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.357492  5113 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:30.391390  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.375072  5320 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:30.387804  5141 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.384877  5295 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:30.406718  1003 raft_consensus.cc:2393] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:30.407047  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.407757  1601 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:30.393558  5363 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.384186  5168 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.402729   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:30.426306   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:30.399870  5320 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:30.446091   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.453965   999 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:30.407467  5141 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:30.392987  5142 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:30.421602   968 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.462634   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:30.455727   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.405642  5295 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:30.467141   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.415336  5131 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.477433   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.472985   968 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.478765   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.475371   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.479864   995 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:30.480791   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:30.481568   922 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:30.481568   929 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:30.484572   929 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:30.458539  5320 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:30.485446   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:30.453361  5294 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:30.503526  5294 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:30.504031  5294 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.508354  5156 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:30.457516  5275 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:30.496515  5214 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.493502  5381 raft_consensus.cc:3055] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:30.524243   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.511020  5156 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.521814   922 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:30.499321  5153 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.477666  5295 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.470728  5363 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:30.524772   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 51769.5us
I20260709 13:14:30.458263  5316 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:14:30.542379   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.554086   921 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:30.405401  5347 consensus_queue.cc:237] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.562227   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.550017   968 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.549255   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.521806  5113 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.522367  5275 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:30.564379   921 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:30.545454  4940 consensus_meta.cc:287] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.001s
I20260709 13:14:30.535761  5381 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:30.582453   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.519668  5176 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:30.587221   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.595997   929 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:30.558483  5317 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.564435  5156 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:30.602892   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.604558   968 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:30.575753  5223 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.584474  5275 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:30.579185  5170 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.624342   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.596856  5176 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.633114   923 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:14:30.633678   923 pending_rounds.cc:85] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 6
I20260709 13:14:30.633988   923 pending_rounds.cc:107] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:14:30.634316   923 raft_consensus.cc:2889] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.593364  5420 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:30.626827   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.615141  5315 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:30.584474  5113 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.647876   998 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:30.590494  5302 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.620725  5363 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:30.592486  4940 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.622717  5276 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.653450   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.656385  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 2125) took 1103 ms. Trace:
I20260709 13:14:30.629531  5156 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:30.637743  5176 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:30.674295  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 338405us
I20260709 13:14:30.672495  1003 rpcz_store.cc:276] 0709 13:14:29.552462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.552883 (+   421us) service_pool.cc:224] Handling call
0709 13:14:30.656175 (+1103292us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54000ea954. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:30.656325 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949996288}
I20260709 13:14:30.682080   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.685714   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.688720   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.693178   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.697770   925 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:30.708168   848 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:30.711203   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:30.711949   923 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:30.659231  5160 raft_consensus.cc:3055] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:30.725373   923 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:30.726464   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:30.728078  1003 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:30.729250  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.685356  5277 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.679154  5156 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:30.730553  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.731406  1596 raft_consensus.cc:1240] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:14:30.732156  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:30.734351   996 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:30.736645  1598 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:14:30.741333  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:30.742087  1597 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:30.735760  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.758175  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.726625  5160 raft_consensus.cc:740] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:30.651767  5381 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:30.760649   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 561418us
I20260709 13:14:30.780648  1597 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:30.828833  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:30.828975   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.825379  1001 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:30.735169  5156 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:30.808902  5361 consensus_meta.cc:287] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
W20260709 13:14:30.766966  5415 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 13:14:30.795929  5279 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.857867  1593 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:30.832543  5187 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:30.858281   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.831205  5168 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.828195  5214 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.781152  5170 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.862040   969 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:30.835124  5316 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:30.785573  5294 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:30.866832  4940 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.874146  5214 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.865044  5187 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.886668   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.887390  5425 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:30.888741  5160 consensus_queue.cc:260] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:30.897105  5384 consensus_meta.cc:287] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.003s
I20260709 13:14:30.898053  5269 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.832754  5131 consensus_queue.cc:1059] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:14:30.856025  5188 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:30.862879  5276 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:30.910043   969 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.912104   922 raft_consensus.cc:2410] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:30.864293  5279 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.924270   969 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:30.911164  5243 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.910812  5384 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.916460  4940 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:30.933130  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.919389  5160 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.857489  5320 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.941835  5384 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:30.942816  5275 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.943296  5320 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:30.944020  5156 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.945698  5176 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.947448  5294 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.928861  5302 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:30.907439  5214 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.905836  5316 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:30.957509   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 155081us
I20260709 13:14:30.950028  5315 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:30.959506   921 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:30.948480  5295 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:30.962020   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.962812   848 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:30.963653   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.964921   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.969636   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:30.969877   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.972913   853 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:30.974848   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:30.976106   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.976334   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.976764   997 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:14:30.978124   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:30.979355   929 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.980849   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.982148   926 raft_consensus.cc:1240] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:30.960860  5315 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:30.983594  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:30.984323   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.985504   846 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:30.985975   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:30.988909  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:30.990139  1003 raft_consensus.cc:2368] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 11. Current term is 12.
I20260709 13:14:30.992769   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:30.993507   995 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:30.968640  5243 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.993889  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:30.930811  5141 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:30.960704  5384 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:31.014663  1593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 719) took 1090 ms. Trace:
I20260709 13:14:30.988683  5315 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.020737  1567 leader_election.cc:400] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 11. Current term is 12.
I20260709 13:14:31.021242  1567 leader_election.cc:403] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:14:31.015165  1593 rpcz_store.cc:276] 0709 13:14:29.923964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.925915 (+  1951us) service_pool.cc:224] Handling call
0709 13:14:31.014462 (+1088547us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54007102d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:31.014593 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990528256}
I20260709 13:14:31.022644  1599 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 14: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:31.028856   995 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:30.987708  5430 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.027550   929 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:31.006932  5243 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:31.038434  5437 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:14:31.038767  5243 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.040738   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.031024  5425 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:31.046947  5316 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:31.041806  5430 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.049542  5430 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.052193  5214 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.053908  5141 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.056136  5156 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.056454  5295 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.057184  5275 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.033630  5198 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.028759  5426 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:31.059417  5426 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.059791  5426 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:31.064153  5279 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.067121   929 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:31.069337   846 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
W20260709 13:14:31.082072  5422 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.348s	user 0.000s	sys 0.000s
I20260709 13:14:31.083410   855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2467) took 1116 ms. Trace:
W20260709 13:14:31.084444  5435 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:14:31.083822   855 rpcz_store.cc:276] 0709 13:14:29.966838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.288894 (+322056us) service_pool.cc:224] Handling call
0709 13:14:30.289349 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:30.289795 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:30.289831 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:30.330913 (+ 41082us) log.cc:844] Serialized 26 byte log entry
0709 13:14:30.573991 (+243078us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:30.574054 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.574158 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.081456 (+507298us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:31.083020 (+  1564us) raft_consensus.cc:1685] finished
0709 13:14:31.083091 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.083350 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1115,"raft.queue_time_us":31778,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3403,"spinlock_wait_cycles":96092544,"thread_start_us":213477,"threads_started":1,"wal-append.queue_time_us":320717}
I20260709 13:14:31.089237   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2448) took 1205 ms. Trace:
I20260709 13:14:31.089797   847 rpcz_store.cc:276] 0709 13:14:29.883635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.883759 (+   124us) service_pool.cc:224] Handling call
0709 13:14:29.907394 (+ 23635us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:14:31.088992 (+1181598us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54002b8c54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:31.089156 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472591872}
I20260709 13:14:31.087325   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40956 (request call id 5) took 1196 ms. Trace:
I20260709 13:14:31.097082   927 rpcz_store.cc:276] 0709 13:14:29.891140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.893782 (+  2642us) service_pool.cc:224] Handling call
0709 13:14:30.617850 (+724068us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54003f8a20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02abc520 00007f0d02b39652 00007f0d07b0760a 00007f0d0108d7d4 00007f0d0108d762 00007f0d0108d691 00007f0d0108d60c 00007f0d0108c8c2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:31.086858 (+469008us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54003f8a20. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02aba5e7 00007f0d02b39c68 00007f0d02b3984a 00007f0d07b0760a 00007f0d0108d7d4 00007f0d0108d762 00007f0d0108d691 00007f0d0108d60c 00007f0d0108c8c2 00007f0d00e7a48c
0709 13:14:31.087221 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241577216}
I20260709 13:14:31.110796  5198 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 11. Current term is 12.
W20260709 13:14:31.115855  5444 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.080s	user 0.003s	sys 0.000s
W20260709 13:14:31.119458   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.118901  5315 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.137573  5426 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.143079  4993 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.155637  5295 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.156271  5295 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.156622  4929 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.158048  5426 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.158443   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.159705  5315 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.159945   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:31.149117  5388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.014s	sys 0.025s
I20260709 13:14:31.161137   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.161232  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.158094  5223 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.164469  5223 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.166087  5285 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.166574  5223 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.166813  5359 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.167508   818 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.168377  5243 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:31.168902  5243 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.169834  5243 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.176100  5196 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:31.178813  5417 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.481s	user 0.000s	sys 0.001s
I20260709 13:14:31.158383  5295 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.168738  4993 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:31.180447   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.186251   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.186894   997 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:31.187898   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.188951   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:31.180752  5388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.014s	sys 0.025s
I20260709 13:14:31.183373  5279 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:14:31.193555   818 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.193918  5279 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:31.194335  5279 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.195569  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.197865  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.198472  1601 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
W20260709 13:14:31.198999   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.199759  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.183094  5359 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:31.200806  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:31.201398   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.205013  1004 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:31.206355  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.202061   969 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:31.208686  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.209344  1000 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:31.195096  5279 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.211094  5279 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.212574  5155 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:31.213150  5155 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.214013  5155 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.217007  1000 raft_consensus.cc:2468] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:14:31.218001  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.219129   995 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:31.220140   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.196890  5355 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:14:31.204799  5374 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:31.197340  5196 consensus_queue.cc:1059] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:31.205389  5359 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:31.190508  5295 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.207823   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.223506   997 raft_consensus.cc:2410] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:31.225014   995 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:31.224059  4780 consensus_queue.cc:1059] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:14:31.228705  1000 raft_consensus.cc:2393] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:31.229519  5355 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:31.235859  4780 raft_consensus.cc:3055] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:31.236308  4780 raft_consensus.cc:740] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.236392  4993 raft_consensus.cc:3055] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:31.236891  4993 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.237246  4780 consensus_queue.cc:260] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.237672  4993 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.238883  4993 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:31.240278   896 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.242697   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.231163  5374 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.241503  5377 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:31.254863  5302 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:31.259143  5374 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.241840  5328 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:31.259954  5328 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.260322  5328 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.254289  5425 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:14:31.255187   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:31.263279   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.265586   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.266306   923 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:31.267299   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.268025   921 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:31.268931   923 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:31.268805   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.230899  5384 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:31.272454  5384 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.273119   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.273775   847 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 5.
I20260709 13:14:31.258455  5377 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.277884  5377 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.234660  5359 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.271847   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.272871  5384 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.258731  5302 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.244272  5214 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.283946   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.284426  1000 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:31.240737  5295 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.279304  5285 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:31.281049  5315 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.293762  5381 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:31.279439  5355 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:31.295081  5196 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:31.301474  5302 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:31.301667  5279 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:14:31.308967  5355 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.310653  5285 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:31.311335  5196 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.319483  5355 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:31.321105  5196 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.322475  5277 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:31.343922  1002 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:31.347283   896 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:31.347847   896 leader_election.cc:403] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:14:31.348839   995 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:31.328744  5277 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.350221   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.348189  5285 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:31.360016  5277 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.361140  4780 raft_consensus.cc:3060] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.361707  5277 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.362711  5359 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.365437  5359 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:31.366011  5440 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.329s	user 0.003s	sys 0.000s
I20260709 13:14:31.366334  5295 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:31.348869  5414 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.725s	user 0.003s	sys 0.001s
W20260709 13:14:31.368080  5418 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.668s	user 0.000s	sys 0.003s
I20260709 13:14:31.377429   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.379763   854 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:31.382553  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:31.386426  5424 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.386974  5424 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.388939   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:31.389560  5457 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:31.389920   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.390413  5426 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:31.376776  5196 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.390797  5426 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:31.391223  5426 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:31.389169  5424 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.392894  5196 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:31.395605   821 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.395449  5295 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:31.396521  5196 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:31.397317  5176 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.397128  5196 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.399349  5196 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.400020  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:31.401286  5241 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:31.401801  5241 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.404527  1599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 727) took 1051 ms. Trace:
W20260709 13:14:31.406308   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.407799  5241 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.409271  5241 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.409912  5241 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.410557  5377 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.412220  5328 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.414529  5328 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.415037  5456 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:14:31.419384   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:31.421757  5411 log.cc:927] Time spent T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.830s	user 0.002s	sys 0.000s
I20260709 13:14:31.375762  5455 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.424561  5294 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:31.424578   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.425441   847 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.431962   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.433996   852 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.434336   847 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:31.427956   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.404937  1599 rpcz_store.cc:276] 0709 13:14:30.353420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.353653 (+   233us) service_pool.cc:224] Handling call
0709 13:14:31.404316 (+1050663us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540024ead4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:31.404469 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500541312}
I20260709 13:14:31.440037  1001 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:31.441433   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52588.8us
I20260709 13:14:31.442615   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.446571  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.446803   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:31.447731   821 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.449054  1567 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.450733  1001 raft_consensus.cc:1240] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:31.457453   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.464947  5462 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:31.465507  5462 raft_consensus.cc:697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:31.467628   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.467927  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.468484   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.467927   995 raft_consensus.cc:1240] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:31.468143  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.470506   929 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:31.469697   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.427546  5144 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:14:31.475035  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.475338  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.476059  5377 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.476790  5377 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:31.477105  5377 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:31.477440  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:31.478638   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.477897  5377 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.450395  5214 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.450598  5322 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:31.482117  1600 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:31.482378   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.483014   846 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 7.
I20260709 13:14:31.483832   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.463968  5381 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:14:31.484114   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:31.473325  5196 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:31.471177  5280 raft_consensus.cc:2804] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:31.488070  1004 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:31.476052  1597 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:31.466599   852 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:31.489079  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.489866   852 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:31.490177  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.491020   852 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:31.491160   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.492158   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.492687   997 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:31.494427   819 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.496271   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.477687  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.504666  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:31.508139   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.512192   921 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
W20260709 13:14:31.514331   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 44749.8us
I20260709 13:14:31.515108  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.516011  1004 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.515936   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:31.520797   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 102699us
I20260709 13:14:31.521337  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.522742   997 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:31.523826   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.525094   997 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:31.526650   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.526752   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.529100   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:31.529958   997 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.531175   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.532078   921 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:31.387352  5384 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.535413   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:31.537231   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.538899   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.542701   926 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.544126  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:31.544780  1601 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.546583  1598 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:31.544780  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:31.547732   818 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.545567  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:31.554780  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.555518  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.556238  1000 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
W20260709 13:14:31.556732   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.555420   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:31.558349   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.561785   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.562386   923 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.559461   819 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:31.487461  5196 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.564191  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:31.520696  5322 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:31.520782  5328 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:31.538445  5359 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:31.613112   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.606288   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:31.628242   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:31.609905   926 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:31.610580  1601 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:31.609943  1004 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:31.414458  5426 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.613773  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.547997  5280 raft_consensus.cc:697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:31.630914  1000 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:31.599385  4993 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:31.466420  5462 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.558499  5241 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.555380  5269 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.619385  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.553519  5320 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.672535  5320 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.673732  5426 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:31.568513  5279 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:31.674537  5196 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.630193   818 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:31.676122   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46012.4us
I20260709 13:14:31.568248  5055 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:31.678186  5035 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.679508  4780 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:31.658438   923 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:31.652722  5267 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.659539  5463 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.662667   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.682005   995 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:31.664381  5464 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:31.541815  5302 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:31.668181  5459 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:14:31.680843  1593 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:31.682138   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 228083us
I20260709 13:14:31.680038  5320 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.612111  5322 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:31.690634   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.684091   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.692281   928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42382 (request call id 2225) took 1400 ms. Trace:
I20260709 13:14:31.692610   928 rpcz_store.cc:276] 0709 13:14:30.291516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.294971 (+  3455us) service_pool.cc:224] Handling call
0709 13:14:31.692112 (+1397141us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b54003282d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:31.692228 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16266496}
I20260709 13:14:31.694245   997 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:31.687630   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 2199) took 1139 ms. Trace:
I20260709 13:14:31.694929   998 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:31.695214   922 rpcz_store.cc:276] 0709 13:14:30.548784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.548955 (+   171us) service_pool.cc:224] Handling call
0709 13:14:31.687403 (+1138448us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400319f54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:31.687554 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790433152}
I20260709 13:14:31.688527   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 683) took 1820 ms. Trace:
I20260709 13:14:31.696348   850 rpcz_store.cc:276] 0709 13:14:29.870920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.895279 (+ 24359us) service_pool.cc:224] Handling call
0709 13:14:30.394371 (+499092us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b5800001298. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b18fc0 00007f0d07b03929 00007f0d07b033d2 00007f0d0107abd4
0709 13:14:30.394615 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.198390 (+803775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.198421 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.198461 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:31.198490 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.688315 (+489825us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b540041f0d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:31.688365 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.688479 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853186176}
I20260709 13:14:31.698244   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.699234   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.691653  5055 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:31.693501  5322 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:31.701828  5055 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.703861  5176 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.705034  5176 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.707011  5302 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.707719   927 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:31.708249  5377 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.682137  4780 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.710997  4780 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.722025   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.724538   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.699750   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:31.746812   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 187357us
W20260709 13:14:31.748087   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 48399.1us
I20260709 13:14:31.748467   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.751351   969 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:31.759533   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.762245   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.765139   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.767463   923 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:31.768365   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.682497  4993 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.769487   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.624933  5355 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.687258  5168 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:31.770105   928 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:14:31.805676   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.774832   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:31.815436   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 67087.4us
W20260709 13:14:31.794704   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70255.4us
I20260709 13:14:31.774190   927 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:31.816859   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.773834  4993 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:31.822084   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.755056  5131 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.820202   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.769474  5315 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:31.822851  5241 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:31.858189  1003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155790402004e4f930983841dfd8c64"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:31.859040  1003 raft_consensus.cc:606] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:31.827095   846 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:14:31.828680  5421 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 1.144s	user 0.003s	sys 0.000s
I20260709 13:14:31.830538   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.830799   926 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:31.831475   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:31.863868   923 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 14: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:31.832476   924 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:31.833094   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.865494   930 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.834921   999 raft_consensus.cc:1240] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:31.817770  5431 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:31.837739  5460 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:14:31.822508  5155 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:31.849548   852 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:31.774168  5168 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.754474  5424 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:31.769321  5363 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:31.861616   849 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 15: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:31.862619   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 305973us
I20260709 13:14:31.870642   928 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:31.838896   927 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:31.877687   846 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:31.772392  5328 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:14:31.885207   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:31.871796  5243 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.856011  5315 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:31.883456   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:31.858189  5241 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.886909  1003 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:31.889369   930 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:31.929651  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:31.892426   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 75668.3us
I20260709 13:14:31.916723   846 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:31.891252  5431 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:31.919924  5384 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:31.863189  5280 consensus_queue.cc:237] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.923699  5363 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:31.925283  5355 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:31.932799   967 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:31.927716   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:31.937440   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64525.1us
I20260709 13:14:31.934859  4940 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.926708  5279 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:31.928638  5243 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.928555  5315 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:31.925734  5455 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
I20260709 13:14:31.880599   852 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:31.950378   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:31.949985   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:31.928069  5456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.004s	sys 0.018s
I20260709 13:14:31.932193  5241 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.913307  5168 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:31.964213   855 raft_consensus.cc:1275] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:14:31.964870   855 pending_rounds.cc:85] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 8
I20260709 13:14:31.965193   855 pending_rounds.cc:107] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:14:31.965479   855 raft_consensus.cc:2889] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:31.923480  5322 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:31.947950  5328 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.974433   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:31.960934  5456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.004s	sys 0.018s
I20260709 13:14:31.933107  5468 consensus_queue.cc:1059] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:14:31.976922  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 357578us
I20260709 13:14:31.959975  5455 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:31.979825   930 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:31.960651  5279 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:31.944362  5176 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:14:31.944061  5347 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:14:31.949222  5355 consensus_queue.cc:1059] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:14:32.002756   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.004462   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.003959  5431 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.005903   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.006652   853 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:32.009234  5243 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.009903  5243 raft_consensus.cc:3055] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:32.009356  5279 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.010404  5243 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.019855   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:31.992601   921 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.994608   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:32.021734   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:31.997061  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.996970  5374 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:14:32.020299  5294 consensus_meta.cc:287] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.003s
I20260709 13:14:31.982846  5168 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:31.975523  5472 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.986580   930 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:31.977509  5456 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:32.023855   853 raft_consensus.cc:2468] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:32.036095   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:32.037350   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 687) took 1342 ms. Trace:
I20260709 13:14:32.044991   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:32.046443   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.042850   854 rpcz_store.cc:276] 0709 13:14:30.695033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.695213 (+   180us) service_pool.cc:224] Handling call
0709 13:14:30.722740 (+ 27527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.380221 (+657481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.036984 (+656763us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b5400071054. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:32.037263 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510438144}
I20260709 13:14:32.045974   853 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:32.031149  5113 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:32.029858  5455 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:32.054500   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:31.995381  5426 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:14:32.055014   921 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:14:32.035432  5294 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.985431  4993 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.029569  5160 consensus_queue.cc:1059] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:14:31.991858  5363 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.031270  5243 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.127780  5243 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.079115  5455 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:32.136610  5435 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:32.136305  5456 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.168818  5455 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:32.212944   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.214201   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 349130us
I20260709 13:14:32.217610   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:32.220398   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.220476   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:32.229673   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.230599   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.232241   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.230635   853 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:32.234514   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.245572   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.250396   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.233111   850 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:32.254091   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.255505   850 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:32.256067   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 315322us
I20260709 13:14:32.257524   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.261230   853 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:32.261370   967 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:32.263154   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 306340us
I20260709 13:14:32.269138   896 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:32.273221   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:32.274349   998 raft_consensus.cc:606] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:32.276917  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:32.277905  1001 raft_consensus.cc:2368] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.278937  5496 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.283295  5496 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:32.288192   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.288419   896 leader_election.cc:400] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.288851   896 leader_election.cc:403] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:32.288664  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:32.289558  1002 raft_consensus.cc:606] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:32.290035  1002 raft_consensus.cc:614] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:32.290212   853 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:32.296432  1599 raft_consensus.cc:1240] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:32.298054   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.297976  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:32.299996  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:32.300451  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:32.305369  4993 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.305658  5113 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:32.305737  5497 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:32.306861  5113 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.308132  5131 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:32.308287   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.309238  1593 raft_consensus.cc:1240] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:32.319515   928 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:32.309206  5131 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.321063  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:32.321980  1598 raft_consensus.cc:2368] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.326468  5280 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:32.170271  5322 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.330111  5302 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:32.330891  5302 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:32.337802   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64739.1us
I20260709 13:14:32.345397  5302 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.344718  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:32.327189  5280 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.352911  5468 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:32.355229   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.355459  5366 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.308056  4993 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:14:32.365212  5468 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:32.365924  5468 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.311667  5170 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:32.357646  1002 raft_consensus.cc:606] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:32.375936  5113 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.376468  5280 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:32.358961  5131 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:32.377673  5431 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:32.378288  5431 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:32.379573  5485 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:32.330128  5320 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:14:32.385805   929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 2220) took 1119 ms. Trace:
I20260709 13:14:32.369625  5468 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.343771  5497 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:14:32.393308  1594 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:32.386328   929 rpcz_store.cc:276] 0709 13:14:31.266182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.266378 (+   196us) service_pool.cc:224] Handling call
0709 13:14:31.266828 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:31.470912 (+204084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.385122 (+914210us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54004470d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:32.385554 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467726080}
I20260709 13:14:32.398958  5424 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:14:32.396085   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:32.393342  5499 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:32.409556  5499 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.416390  5187 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.418475  5315 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.419102  5223 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:32.419713  5223 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:32.421852  1003 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:32.422667   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 104906us
I20260709 13:14:32.423439  5499 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:32.423251   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 494620us
I20260709 13:14:32.423779  5223 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.425406  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:32.425326  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:32.427253  5468 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.423784   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.426697   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.428138   996 raft_consensus.cc:2368] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.429069   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:32.432405  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:32.434190  1593 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.438881  5223 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.445284  5381 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:32.445848  5381 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:32.447316  5381 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.451284  5381 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.429987  5498 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:32.456427  5498 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:32.456830  5498 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.457906  5498 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.462126   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51144 (request call id 2141) took 1270 ms. Trace:
W20260709 13:14:32.462877   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.462615   996 rpcz_store.cc:276] 0709 13:14:31.166547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.166691 (+   144us) service_pool.cc:224] Handling call
0709 13:14:32.394019 (+1227328us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800121598. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cffa1ab71 00007f0cffa1aa22 00007f0d08933c09 00007f0d07a8dee0 00007f0d07b570ab 00007f0d07b57131 00007f0d07b050a0 00007f0d07b04984 00007f0d0107f6d4 00007f0d0107f662
0709 13:14:32.436790 (+ 42771us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29907840}
I20260709 13:14:32.464164   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.465608  5355 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:32.466120   849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2490) took 1034 ms. Trace:
W20260709 13:14:32.466862   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 54554.9us
I20260709 13:14:32.467834  5275 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:32.467960   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:32.469600   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46053.2us
I20260709 13:14:32.470005  5431 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.471431  5495 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:32.472826  5495 consensus_queue.cc:1059] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:32.473521  5275 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.473912  5495 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:32.474392  5495 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:32.475770  5495 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.476378   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.477581  5187 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:32.479995  4940 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:32.451107  5241 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.481878   852 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:32.483871  5241 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.459342  5498 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.483969   929 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:32.497367   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.497191  5241 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:32.485801   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:32.500720  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.485805   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:32.503062   854 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:14:32.504416   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78442.5us
I20260709 13:14:32.506385   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.466591   849 rpcz_store.cc:276] 0709 13:14:31.431314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.431444 (+   130us) service_pool.cc:224] Handling call
0709 13:14:32.465682 (+1034238us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540019e6d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:32.465930 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902135552}
I20260709 13:14:32.508304  4993 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.508469  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:32.509105  4993 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.509558  1601 raft_consensus.cc:2393] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:32.509754  4993 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.511322  5431 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:32.513223   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.514588  5472 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:32.515153  5499 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.515859  5499 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.518395   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.519361  5155 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:32.519941  5155 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.520385   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.488085  5495 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:32.492336   846 raft_consensus.cc:2410] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:32.516577  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.524287  4993 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.525148  5374 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:32.527117  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:32.528200  1601 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 5.
I20260709 13:14:32.537973   921 raft_consensus.cc:1240] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:32.527359   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.492561   894 leader_election.cc:400] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.495690   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.542970  1593 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:32.547472   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.552945  5473 raft_consensus.cc:3055] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:32.553526  5473 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:32.553192  5267 consensus_queue.cc:1059] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:14:32.555280  5267 raft_consensus.cc:3055] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:32.558202   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:32.561250   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 68872.6us
I20260709 13:14:32.567628  5322 consensus_queue.cc:1059] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:32.568113  5384 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:32.568773  5384 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:32.569896  5384 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.573453  5214 raft_consensus.cc:3055] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:32.573889  5214 raft_consensus.cc:740] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:32.551376  4993 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:14:32.596004  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 43953.4us
I20260709 13:14:32.550307  5498 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:32.558591  5499 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:32.601485  1593 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:32.562023  5267 raft_consensus.cc:740] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.603796   846 raft_consensus.cc:2368] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
I20260709 13:14:32.604960   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.604892  1601 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:14:32.606402   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.577203  4993 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.607743  1594 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:14:32.555152  5473 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.630395  5473 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:32.631398   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.633857  5277 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.634788  5277 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.636989   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.637728   928 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:14:32.620249   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 134679us
I20260709 13:14:32.621706  1599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 469) took 1147 ms. Trace:
I20260709 13:14:32.640270  5498 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:32.640467  1599 rpcz_store.cc:276] 0709 13:14:31.474307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.482747 (+  8440us) service_pool.cc:224] Handling call
0709 13:14:31.483023 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.296984 (+813961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.621373 (+324389us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b5400591d54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:32.621614 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869104384}
I20260709 13:14:32.638149  5277 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:32.639355  5267 consensus_queue.cc:260] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.644371   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.645107   928 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:32.645424  5497 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:32.646039  5497 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.646795   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:32.648325  5359 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:32.648720  5359 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.649295  5359 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.622584  5214 consensus_queue.cc:260] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.651548   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:32.627637  5374 consensus_queue.cc:1048] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:32.621932   821 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.653277  5241 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:32.644932  5267 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:32.633925   926 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:32.627954  5055 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:32.636794  5461 raft_consensus.cc:3055] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:32.655959  5461 raft_consensus.cc:740] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.656082  5055 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.657033  5461 consensus_queue.cc:260] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.658543  5461 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.611008  4993 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:32.668226  5315 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:32.670331  5503 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.255s	user 0.000s	sys 0.004s
I20260709 13:14:32.654043   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:32.718899   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 48264.8us
I20260709 13:14:32.745280  4780 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:32.745595   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 91701.3us
I20260709 13:14:32.746057  4780 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:32.746730  4780 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.747807   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:32.749183  5277 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:32.751796  5277 consensus_queue.cc:1059] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:32.753232  5498 raft_consensus.cc:3055] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:32.753768  5498 raft_consensus.cc:740] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:32.756471   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.758026   969 leader_election.cc:400] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
I20260709 13:14:32.764751   969 leader_election.cc:403] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:32.736331  5055 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.755148  5498 consensus_queue.cc:260] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:32.784943   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.787185  1564 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:32.788614  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 181304us
I20260709 13:14:32.797988  4993 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:32.799685   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.804198  4993 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.812773   921 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:32.817265   847 raft_consensus.cc:1240] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:32.819154   853 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:32.820184 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:32.820650  5477 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:32.820930   819 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:32.821318  5477 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:32.822583  5477 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.823464  5131 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:32.824249  5131 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.823515  5277 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:32.835223   848 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:32.836195   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:32.826859  5131 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:32.833249   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 96837.9us
W20260709 13:14:32.840910   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 44287.2us
W20260709 13:14:32.842453   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.848280  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.835443  5277 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:32.861181  5176 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:32.861891  5176 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.836135  4780 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.872267  1564 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.850003  5498 raft_consensus.cc:3060] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:32.877384   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:32.877595  5073 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:14:32.879148  5277 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.881174  5155 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.849810  5144 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.886341  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.884768   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:32.837877  5241 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:14:32.903852  5276 raft_consensus.cc:993] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:32.904095  5363 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.905458  5198 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:32.906003  5198 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:32.907104  5198 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:32.908448  5198 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.911840   896 leader_election.cc:400] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.912227   896 leader_election.cc:403] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:32.912843  5144 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:32.914072   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.884262  5381 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:32.915164 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:32.926024   929 raft_consensus.cc:1275] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:14:32.928047   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.940155   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.948439   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:32.924118  5499 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:32.939564  5113 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.958176   996 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:32.964820   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 119452us
I20260709 13:14:32.944698  5276 raft_consensus.cc:1081] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:32.972901  5456 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:32.973477  5294 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:32.976099  5155 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:32.979552   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:32.981751   995 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:32.983616   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:32.985541  1000 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:32.986892  5359 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:32.990757  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:32.950568  5223 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:14:32.992295  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.992543  5497 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:32.992434  4780 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:32.981787  5512 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:14:32.971750  5176 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:33.020754  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:33.029923  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.032557   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 49068.7us
I20260709 13:14:33.033423   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:33.034228  1567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.018435  5113 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:33.044525   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:33.045828   999 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:33.046234  5214 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.048293  5473 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:33.002841  5294 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.054937  5363 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11.
W20260709 13:14:33.057394   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.061838   997 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:14:33.069586  5510 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:33.070485  5363 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:33.071679  1003 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:33.071267  5363 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.074047  5363 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:33.038661  5381 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:33.081885  5381 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.088760   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.089592   924 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:33.092880  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.116032  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.116032   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:33.118772  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 232503us
I20260709 13:14:33.123996  5286 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:14:33.126559   927 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:33.130004  5275 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:33.130693  5275 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:33.130089   929 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:33.140394   847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2498) took 1522 ms. Trace:
I20260709 13:14:33.138932  1002 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:33.140811   847 rpcz_store.cc:276] 0709 13:14:31.600197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.835702 (+235505us) service_pool.cc:224] Handling call
0709 13:14:31.836035 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.817846 (+981811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.113584 (+295738us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54000ca8d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:33.122823 (+  9239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078819712}
I20260709 13:14:33.143918   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:33.095841  5456 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:14:33.147146  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:33.151307   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:33.153394  5286 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:33.159883  5286 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.160247  5286 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:33.173005   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 350512us
I20260709 13:14:33.175783  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:33.177477  1594 raft_consensus.cc:2393] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:33.179404  1601 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:14:33.189883   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46062.5us
I20260709 13:14:33.179960  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:33.195937   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.198910   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:33.203001  5113 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:33.203724  5363 consensus_queue.cc:1048] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:33.207815  5381 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.208861  5187 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:33.209424  5187 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:33.209884  5187 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.202039  5461 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:33.213037  5461 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.215021   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.217446   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.217901  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.217101   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.233271   997 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:14:33.231700   998 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:33.234440   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:33.239753   847 raft_consensus.cc:1275] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:14:33.240250   847 pending_rounds.cc:85] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 11
I20260709 13:14:33.241919   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.215572  5461 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:33.240487   847 pending_rounds.cc:107] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:14:33.252905   847 raft_consensus.cc:2889] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.245795   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.250751  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:33.254118   996 raft_consensus.cc:2410] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:14:33.256336   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.239024  4993 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:33.267405   924 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:33.276410  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:33.279815   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.203773  5113 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.305814   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.299816   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.235006  5073 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:14:33.313469  1601 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:33.319252   995 raft_consensus.cc:2393] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 8: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:33.324859   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.326750   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.335871   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.336707   847 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:33.338301  5501 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:33.338678  5170 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:33.341640  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:33.346691  1001 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:33.325666  5286 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.339668  4940 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Committing config change with OpId 10.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:33.347712   997 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:33.322491  5498 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:33.362370   850 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:33.366178   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.328519  5176 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:33.355468  5055 raft_consensus.cc:993] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:33.330368  5381 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:14:33.368896  4993 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:33.415724  4993 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.417948  4993 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.418771   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:33.420513   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 82843.4us
I20260709 13:14:33.382177   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.421320   995 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:33.383002   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.387876  5498 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.425266  5498 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:33.425503  5526 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:14:33.426389   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:33.392232   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:33.427167   923 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.352998  5294 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.354533  5359 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.349583  5286 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:33.428798   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 48380.1us
I20260709 13:14:33.429267  5363 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:33.430328   968 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85bbb1b886474a538a2f1a32dd91b560. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:33.432219  5285 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:33.399973  5160 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.400683  5497 raft_consensus.cc:1081] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:33.401378  5144 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.442294  5160 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.402405  5113 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.408733  5275 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.449012  5275 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.449934   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.450770   929 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:33.456116   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.458279  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:33.459049  1601 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:33.459784  5160 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:33.413290  5176 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.460377   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:33.461993  5497 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.462883   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.383317   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.467360  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:33.488813   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.505116   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:33.475836  5315 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:33.508361   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:33.525058  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57778.2us
I20260709 13:14:33.528867  1567 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.525206  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.530335  1001 raft_consensus.cc:2410] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:33.510942   846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9155790402004e4f930983841dfd8c64"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:33.531615   846 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:33.531984   846 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.533757   996 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:33.538141  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.541574  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.550160  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:33.545982  5522 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.222s	user 0.001s	sys 0.001s
I20260709 13:14:33.505100  5294 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:33.561115   995 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:33.564380   923 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:33.484328  5280 consensus_queue.cc:1048] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:14:33.543666  5498 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:33.538010  5477 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:33.575848   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.543864  5499 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:33.588477   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.566198  5374 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.582672   818 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:33.618798   996 raft_consensus.cc:2393] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:33.568207  5294 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.575254  5286 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.568140  5519 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.532931  5497 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.567616  5160 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:33.582448  5498 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:33.625360  5294 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:33.623657  5322 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:33.642367  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 92319.7us
I20260709 13:14:33.626814  4993 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:33.649036   848 raft_consensus.cc:1240] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:33.650390   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 187677us
W20260709 13:14:33.661932  5531 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:14:33.667752   753 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 7 to 10, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:33.668387   854 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Committing config change with OpId 10.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:33.643324  5160 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.659252   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.638821  5498 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.674495   855 raft_consensus.cc:2410] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:33.677929   846 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.688723   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.691792   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.640185  5477 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:33.694768  4993 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:33.696316   923 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:14:33.696684  5530 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:14:33.697352  1003 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:33.698910  5160 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.698814   852 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:14:33.702558   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.715327   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.728598  5456 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:33.729055   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.730227  5456 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:33.730890   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:33.634048  5374 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.703243   753 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 11 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:33.721385   846 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.722092  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.716044   896 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:33.727191   852 pending_rounds.cc:85] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
I20260709 13:14:33.740527   852 pending_rounds.cc:107] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:33.740860   852 raft_consensus.cc:2889] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.735535   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.637058  5286 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:33.756361   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.756798   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:33.757580   923 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.653229  5322 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.760828  1004 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:33.765935   849 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:33.705140  5384 consensus_queue.cc:1059] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:14:33.711671  5187 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:33.749590  5456 raft_consensus.cc:2804] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:33.793570  1003 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:33.806219   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.736886  5055 consensus_queue.cc:1059] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:14:33.808820  1003 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:33.815222   848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:33.816015   848 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:33.816457   848 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.827459  1003 raft_consensus.cc:1240] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:33.812151  5302 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:33.817692  5384 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:33.839984   849 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:14:33.842724   849 pending_rounds.cc:85] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 8
I20260709 13:14:33.842962   849 pending_rounds.cc:107] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:14:33.843283   849 raft_consensus.cc:2889] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.817072  5456 raft_consensus.cc:697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:33.849434   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 83334.2us
W20260709 13:14:33.844331   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 96610.3us
W20260709 13:14:33.846328   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.856226  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:33.861728   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 71584us
I20260709 13:14:33.863396   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:33.867777   847 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:33.827144  5055 consensus_queue.cc:1243] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 64f2cb5b0e174002811ef197f9eec829 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:33.842309  5302 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:33.842190  5384 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:33.871145  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.875555   852 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:33.869733   847 pending_rounds.cc:85] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Aborting all ops after (but not including) 6
W20260709 13:14:33.865355  5527 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:14:33.891165  1594 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:33.906363   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:33.907051   846 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:33.910991   923 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:33.914793   847 pending_rounds.cc:107] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:33.915280   847 raft_consensus.cc:2889] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.820051  4993 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.865598  5176 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:33.889600  5170 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:33.917964  5538 log.cc:927] Time spent T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:14:33.936442  5328 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:14:33.936692  5384 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.937914  5384 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.938820  5322 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:33.940780  5187 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:33.941612  5532 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
W20260709 13:14:33.918749   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 188009us
I20260709 13:14:33.919482   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:33.924357  5540 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:14:33.920648  5534 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:14:33.925755  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.925994  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.926426  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.889730  5359 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.945514   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.858556  5381 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 13:14:33.934973  5539 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:14:33.935730  5537 log.cc:927] Time spent T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:14:33.935580  5073 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.936610  5456 consensus_queue.cc:237] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:33.924170   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.880966  5302 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:33.926808  4940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39762 (request call id 0) took 8358 ms. Trace:
W20260709 13:14:33.966080  5535 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:14:33.965256 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:33.916438  5497 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:33.921918  5267 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.988195   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:34.021035  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:33.988343  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:33.952215  5187 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.978863  5214 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:34.023808   996 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:34.055966   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.069459   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.046348  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:34.057607   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.117435   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.118140  1001 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:34.118516  1001 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:34.121006   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 187880us
I20260709 13:14:34.122939  5495 raft_consensus.cc:3055] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:34.123119   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.123369  5495 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:34.123711  5286 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.024987  5073 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.114944   848 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.125442  5556 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.057255  5144 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:34.083819  5540 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:14:34.143657  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 155820us
I20260709 13:14:33.983705  5359 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.161310   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:34.026777  5113 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:14:34.111662  5473 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.164004   848 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.138355  5280 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.144022   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.155789  1001 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.145205  5559 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.173190  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.128800  5499 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 13:14:34.190878  5509 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:14:34.129850  5267 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:34.196992  5547 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:14:34.162518  5144 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.144996  5187 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.235466  5187 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
W20260709 13:14:34.203970   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 223119us
I20260709 13:14:34.210265   997 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:34.193467  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:34.237052  1593 raft_consensus.cc:2393] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:34.237769  1003 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:34.192323  5551 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:14:34.176566  5322 raft_consensus.cc:2764] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:34.157200  5556 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.206176  4780 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:34.177358  5055 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:34.193077  5113 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:34.190958  5544 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.242753  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.235709  5144 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.190693  5561 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:34.244211   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 100285us
I20260709 13:14:34.024987  4940 rpcz_store.cc:276] 0709 13:14:25.544965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.545116 (+   151us) service_pool.cc:224] Handling call
0709 13:14:25.565428 (+ 20312us) log.cc:844] Serialized 499 byte log entry
0709 13:14:33.873138 (+8307710us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10457,"raft.run_cpu_time_us":1753,"raft.run_wall_time_us":1749}
W20260709 13:14:34.258769   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 142775us
I20260709 13:14:34.193274  5473 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.242494  5384 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:34.264304   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:34.259639  5315 consensus_meta.cc:287] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.001s
I20260709 13:14:34.191301  5495 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.265251   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.272306  5495 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:34.272648   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.230685  5559 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.277034  5473 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.261847  5113 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:34.278294  5559 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.278091  5302 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:34.278895  5275 consensus_meta.cc:287] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.003s
I20260709 13:14:34.279270   923 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:34.260572  5055 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.280623  5359 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:34.269578   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.271389  5267 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.280751  5556 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:34.259781  5519 consensus_meta.cc:287] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 13:14:34.297947  5519 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:34.226769  5280 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.285992  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.300482  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:34.264173  5384 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.262288  5497 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:34.285022  5550 ts_tablet_manager.cc:933] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.296695  5073 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:34.295312   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 134117us
W20260709 13:14:34.313834   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 190775us
I20260709 13:14:34.301362  1594 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:34.284439  5322 raft_consensus.cc:2764] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:34.318198  5176 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:34.318281  5055 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.320252   849 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:34.336712   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:34.340036   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83906.9us
I20260709 13:14:34.341130   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:34.343890  5322 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.312395  5384 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:34.323206   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.346925  5563 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:34.323252   852 raft_consensus.cc:1240] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:34.285992  5315 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:34.331995  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.264081  5285 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:14:34.266098  5170 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:14:34.278352  5113 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.335354  5302 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.350670  1593 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
I20260709 13:14:34.302012  5359 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:34.312711  5267 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:34.357113   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.357779   895 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:34.257786  5544 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:34.363955   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.368284   855 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:34.354689  5497 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:34.392253  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.393036  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:34.268277  5561 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:34.395511   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45875.6us
I20260709 13:14:34.397630  4780 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:34.418875   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.370370  5359 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.422120   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.396314  5176 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:34.430243  5574 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:34.431533   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:34.413167  5280 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.373594  5267 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:34.436637   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 93026.3us
I20260709 13:14:34.437449  5113 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:14:34.438352   801 auto_rebalancer.cc:629] Failed to schedule move for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c: Illegal state: Replica 3bb5726887244caca146e176cc6bd070 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:34.438267   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.439733   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.440723  4993 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.440871  5575 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:34.444325   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:34.426942  5498 consensus_meta.cc:287] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.000s
I20260709 13:14:34.394768  5579 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.433135  5564 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.438231  5113 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:34.466745  5564 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.462024  5575 raft_consensus.cc:3060] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:34.463418  5267 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.463636  5359 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:34.465803  5562 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:14:34.465904   801 auto_rebalancer.cc:629] Failed to schedule move for tablet 2fb0c412ac6a42fe84d4b2ce85773e26: Illegal state: Replica 3bb5726887244caca146e176cc6bd070 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:34.461826   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:34.435643  5285 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:34.490805   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47033.5us
I20260709 13:14:34.492471  5582 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:34.489317  5285 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.496872  5561 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.436228  5176 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.502420   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.503108   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:34.495048  5384 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.457283  5498 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:34.527864   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66145.7us
I20260709 13:14:34.475529  5579 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:34.539927   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48364.5us
I20260709 13:14:34.541764  5579 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.542178   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:34.543884  5570 log.cc:927] Time spent T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:14:34.548869  5384 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:34.549964   751 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 224) took 11589 ms (client timeout 15000 ms). Trace:
I20260709 13:14:34.550051  5384 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.551694   753 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 6, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 6 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:34.553264  5564 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.502554  5550 tablet_copy_client.cc:323] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:14:34.553341  5176 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.528838  5359 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.505985  5160 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:34.558857  5384 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:34.559828   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 56806.4us
I20260709 13:14:34.561909  5574 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:34.562177  5587 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:34.561396   821 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:34.554533  5560 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:14:34.556351  5187 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.572383   753 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 9 to 12, leader changed from <none> to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:34.581389  5573 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.591678  5501 raft_consensus.cc:993] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:34.594236  5374 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.597589   851 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:34.606463   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45145.8us
I20260709 13:14:34.607837  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.562532  5267 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.615362  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:34.615396  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.561910  5285 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.616374  1599 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:34.617043   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:34.620150  4780 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:34.620621   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.623260   821 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.627485  5578 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:34.630282   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:34.631187  5553 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.510s	user 0.003s	sys 0.000s
I20260709 13:14:34.631794  5578 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:34.632963  5498 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.632704   753 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 8 to 10, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:34.639911  5187 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.644220   753 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:34.645713   753 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 5 to 10, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:34.645241   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:34.608180  5564 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.651489   921 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:34.607673  5587 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:34.652722  5160 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:34.657259   922 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:34.658056   753 catalog_manager.cc:5697] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:34.642427  5187 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.642972  5501 raft_consensus.cc:1081] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:34.662415   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.625844  4993 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:34.689481   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 85579.4us
I20260709 13:14:34.690428   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:34.690799   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:34.617637  5573 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.696566  5573 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.696810   926 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:34.657439  5587 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:34.664382  5575 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:34.700863   850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2531) took 1080 ms. Trace:
I20260709 13:14:34.701020  5267 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:34.701272   850 rpcz_store.cc:276] 0709 13:14:33.620278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.620467 (+   189us) service_pool.cc:224] Handling call
0709 13:14:33.632503 (+ 12036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:33.643920 (+ 11417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.643954 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:33.647079 (+  3125us) log.cc:844] Serialized 26 byte log entry
0709 13:14:33.817842 (+170763us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:33.817895 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.817981 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:34.630501 (+812520us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:34.700566 (+ 70065us) raft_consensus.cc:1685] finished
0709 13:14:34.700590 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:34.700806 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3121,"raft.queue_time_us":6270,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":21637,"spinlock_wait_cycles":19159424,"thread_start_us":106769,"threads_started":1,"wal-append.queue_time_us":278105}
W20260709 13:14:34.556888   751 rpcz_store.cc:269] 0709 13:14:23.010108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.046966 (+ 36858us) service_pool.cc:224] Handling call
0709 13:14:34.549741 (+11502775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:30.129035 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:30.129590 (+   555us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:30.129626 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:14:30.129643 (+    17us) tablet.cc:662] Decoding operations
0709 13:14:30.527304 (+397661us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:30.539686 (+ 12382us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:30.543032 (+  3346us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:14:30.779492 (+236460us) tablet.cc:701] Row locks acquired
0709 13:14:30.779512 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:14:30.779736 (+   224us) write_op.cc:270] Start()
0709 13:14:30.779878 (+   142us) write_op.cc:276] Timestamp: P: 1783602870779675 usec, L: 0
0709 13:14:30.779897 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:14:30.780680 (+   783us) log.cc:844] Serialized 10129 byte log entry
0709 13:14:31.145116 (+364436us) op_driver.cc:464] REPLICATION: finished
0709 13:14:31.208659 (+ 63543us) write_op.cc:301] APPLY: starting
0709 13:14:31.208754 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:31.261954 (+ 53200us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:31.272410 (+ 10456us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:31.426475 (+154065us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:31.426846 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 13:14:31.426891 (+    45us) write_op.cc:312] APPLY: finished
0709 13:14:31.427314 (+   423us) log.cc:844] Serialized 228 byte log entry
0709 13:14:31.427476 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:31.447627 (+ 20151us) write_op.cc:454] Released schema lock
0709 13:14:31.467173 (+ 19546us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:31.477299 (+ 10126us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:31.477329 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3363123,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":63260,"apply.run_cpu_time_us":33943,"apply.run_wall_time_us":268608,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112807,"num_ops":26,"prepare.queue_time_us":156454,"prepare.run_cpu_time_us":47799,"prepare.run_wall_time_us":800549,"raft.queue_time_us":101350,"raft.run_cpu_time_us":7771,"raft.run_wall_time_us":41660,"replication_time_us":365063,"thread_start_us":331058,"threads_started":4,"wal-append.queue_time_us":179688,"wal-append.run_cpu_time_us":14446,"wal-append.run_wall_time_us":1525216}]]}
I20260709 13:14:34.706547  5384 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.676764   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.707263   922 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:34.626112  5214 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:34.707911  5214 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.659185  5160 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.709947  5214 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:34.711318  5294 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 1.019s	user 0.008s	sys 0.004s
I20260709 13:14:34.744181   853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2532) took 1104 ms. Trace:
I20260709 13:14:34.744069  5294 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.744562   853 rpcz_store.cc:276] 0709 13:14:33.639852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.640053 (+   201us) service_pool.cc:224] Handling call
0709 13:14:33.647315 (+  7262us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:33.693399 (+ 46084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.693440 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:33.738323 (+ 44883us) log.cc:844] Serialized 52 byte log entry
0709 13:14:33.852015 (+113692us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:33.852074 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.852182 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:34.553818 (+701636us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:34.743887 (+190069us) raft_consensus.cc:1685] finished
0709 13:14:34.743908 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:34.744122 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20267,"raft.queue_time_us":32739,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":17519,"spinlock_wait_cycles":6979968,"thread_start_us":49240,"threads_started":1,"wal-append.queue_time_us":596745}
I20260709 13:14:34.683228   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.683525   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.684046   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.749670  5131 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:34.750034   894 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.750396  5131 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.753192  5131 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:34.753594  5302 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:34.754005  5302 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.706468  5267 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:34.713717   895 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:34.678671  5592 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:34.709971  5160 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.718627  5544 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.768576   929 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:34.721674  5214 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:34.768154  5592 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:14:34.775182   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 297963us
W20260709 13:14:34.724982  5559 consensus_queue.cc:1175] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ff541db50a49efa03669dfd777fc8e" responder_term: 10 status { last_received { term: 10 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 10 index: 7 } }
I20260709 13:14:34.728579   921 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:34.665534   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.783460   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.734081   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 2140) took 1102 ms. Trace:
I20260709 13:14:34.739010  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.738844  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:34.739540   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.739751  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.739624  5381 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 13:14:34.743275  5576 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.369s	user 0.000s	sys 0.003s
I20260709 13:14:34.746069  5294 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:34.754350  5302 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:34.756026   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.678110   927 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
W20260709 13:14:34.719960   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.777614  5592 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:34.781731   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:34.780752   925 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:34.785486  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.783953   851 rpcz_store.cc:276] 0709 13:14:33.631921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.632041 (+   120us) service_pool.cc:224] Handling call
0709 13:14:33.640939 (+  8898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.618638 (+977699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.733830 (+115192us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54000c99d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:34.734016 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2199579392}
I20260709 13:14:34.787665  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:34.780964   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.796051   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:34.796867   996 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 11: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:34.797796  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.798652  1000 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:14:34.799042  1000 pending_rounds.cc:85] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 6
I20260709 13:14:34.799254  1000 pending_rounds.cc:107] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:14:34.799499  1000 raft_consensus.cc:2889] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.803968  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.792148  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.792353   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.805402  1003 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
W20260709 13:14:34.807299   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 51355.1us
W20260709 13:14:34.809722   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70266us
I20260709 13:14:34.812439   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.827580   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:34.834231   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.835386   821 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:34.837252  5574 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:34.837731  5574 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.838068  5574 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:34.838627   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59220.4us
I20260709 13:14:34.838963  5501 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:34.839704  5557 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:14:34.840525   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.841455   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:34.839424  5501 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.842254  5501 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:34.843954   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.845183   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:34.845306   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.845784   850 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:34.846421   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.847298   849 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:34.847862   849 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:34.848188   849 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:34.804782  5575 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.806836  5563 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.835043   924 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:34.860796  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.865293   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.865584  1000 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:34.859990  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:34.786132  5587 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:34.852377   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.829409  4993 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:34.826638  5214 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:34.867396  1600 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:34.862771   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:34.871156  4993 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:34.866187  1564 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.890815  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.891742  1000 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:34.833751  5544 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:34.857514   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:34.894814  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.897056   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:34.887115  5214 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.831174  5567 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:14:34.906757   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 49349.1us
I20260709 13:14:34.908232  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.909621   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.914693  4780 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:34.915302  4780 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:34.909942  5214 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.897099  4993 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:34.934588  1566 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.914005   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.941596  5578 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:34.866835  5575 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.944537  5575 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:34.944650  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 85bbb1b886474a538a2f1a32dd91b560 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:14:34.945465  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 85bbb1b886474a538a2f1a32dd91b560 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-85bbb1b886474a538a2f1a32dd91b560
I20260709 13:14:34.950045  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.952668   895 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:34.951491  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:34.955680  5574 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:34.957226   855 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 14: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:34.958508  5592 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.960824  1000 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:34.962909  5214 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:34.967150   997 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:34.967653   997 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:34.969866  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:34.970034  5573 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:34.970463  1601 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.972254  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:34.973395   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.973966   998 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:14:34.974771   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:34.975292   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:34.979907   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.980762   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:34.981928   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 43940.2us
I20260709 13:14:34.897694  5572 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:34.869789  5563 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.045825  1000 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
W20260709 13:14:35.060559   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 80719.1us
W20260709 13:14:34.985548  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 51083.1us
W20260709 13:14:34.991765   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.996658  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:35.058126  1004 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:35.061769   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 81280us
I20260709 13:14:35.063062   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:34.911496  5267 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:35.065529  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:35.045984  5592 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:35.070730  5322 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.068023  4780 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.093422   855 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:34.937775  5497 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.095016   850 raft_consensus.cc:2468] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:34.967887  5214 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:35.055985  1600 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:35.100960   846 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:35.093386   997 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.106489  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:35.066416  5498 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.117823  1001 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:35.118901   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:14:35.100617  5547 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:14:35.122329   896 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:34.911137  5544 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.127897   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:35.131250   929 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:35.130733   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:35.097443  5578 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:35.139531  1601 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
W20260709 13:14:35.146152   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 193606us
I20260709 13:14:35.107998  5572 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.970568  5573 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:35.119462  5381 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:35.116196  5267 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:35.170475   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:35.140573  5214 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:35.147912  5565 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:35.184031   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:35.191778   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:35.155155  5572 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.162489  5275 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:35.118039  5497 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.160982  5381 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.198313  1001 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:35.218379  5322 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:35.229707  5574 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.231045  1000 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:35.236749  1001 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51128 (request call id 2207) took 1313 ms. Trace:
I20260709 13:14:35.227401   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:35.191321  5214 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:35.263577   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 99657.1us
I20260709 13:14:35.253414  1001 rpcz_store.cc:276] 0709 13:14:33.923260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.978798 (+ 55538us) service_pool.cc:224] Handling call
0709 13:14:34.724785 (+745987us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b18001b0480. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b7a646 00007f0d02ab458e 00007f0d02b35f9e 00007f0d02b377de 00007f0d02b4327c 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c
0709 13:14:35.089493 (+364708us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b580053c0d0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02a9b0aa 00007f0d02b1d447 00007f0d02b35fe7 00007f0d02b377de 00007f0d02b4327c 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c
0709 13:14:35.236687 (+147194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440539264}
I20260709 13:14:35.254457   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:35.274096  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:35.274824  1002 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.262519  5592 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.276031   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:35.296123   929 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.223826  5519 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:35.219240  5578 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:35.276841  1004 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:35.263288  5539 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:14:35.241007  5381 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.244524  5144 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:35.221418  5557 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:14:35.266050  5113 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:35.272716  5214 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.266222  5322 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:35.303375   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:35.271337  5587 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:35.315764   969 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:35.287812  5294 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:35.338922   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:35.302644  5592 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.318934  5113 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.319406  5578 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.371932  5597 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:35.396203  5597 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.396595  5597 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.396425  5322 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.397588  5578 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.403090   929 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
W20260709 13:14:35.415098   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 101812us
W20260709 13:14:35.417055   849 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 13:14:35.418005   849 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.375782  5294 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:35.308163  5267 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.377640  5595 raft_consensus.cc:2804] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:35.438095  5595 raft_consensus.cc:697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:35.382095  5113 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.376578  5473 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:35.424782  5592 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:35.394199  5574 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:35.428195  5600 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:14:35.371460  5563 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.436609  5497 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:35.300051  5573 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.395077  5565 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:35.450614   849 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.452669  1002 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:35.420346  5359 raft_consensus.cc:2764] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:14:35.506313  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:35.545257  1601 raft_consensus.cc:2368] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
W20260709 13:14:35.549327  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 422634us
I20260709 13:14:35.550384  5572 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:35.553609  5302 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.751s	user 0.009s	sys 0.000s
I20260709 13:14:35.554057  5302 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.555222   923 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 14: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:35.558372  1014 tablet_copy_source_session.cc:215] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:35.558743  5144 consensus_queue.cc:1059] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:35.561412   997 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:35.562191  5603 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 13:14:35.563158  5275 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:35.564716   819 leader_election.cc:400] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
I20260709 13:14:35.565048   819 leader_election.cc:403] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:35.565042  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 447475us
W20260709 13:14:35.542788  5605 log.cc:927] Time spent T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.268s	user 0.001s	sys 0.003s
W20260709 13:14:35.516721  5601 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:14:35.518258  5609 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:14:35.563659  5275 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:35.487486  5544 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:35.578115  5556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.000s	sys 0.010s
I20260709 13:14:35.584297  5617 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:35.465114  5473 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:35.596511   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 342118us
I20260709 13:14:35.601190   853 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:35.600337   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:35.602475   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.602614   849 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
W20260709 13:14:35.595103  5612 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:35.492393  5574 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:14:35.593406  5556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.000s	sys 0.011s
I20260709 13:14:35.532300  5294 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:35.605430   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.547288  5565 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.611514   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:35.612674   929 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:35.615480   969 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:35.623025  5565 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.624151   997 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
W20260709 13:14:35.626734  5561 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:14:35.547576  5595 consensus_queue.cc:237] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.623654  5565 raft_consensus.cc:3055] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:35.636535  5487 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:35.720615  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.722986   849 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:35.754177   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 324695us
I20260709 13:14:35.755913   895 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:35.758291   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 474906us
I20260709 13:14:35.642882  5565 raft_consensus.cc:740] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:35.719316  5487 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:35.800580  5629 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:14:35.811895  5302 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:35.847303  5302 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:35.723776  5590 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:35.795272  5322 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:35.828568   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 397970us
W20260709 13:14:35.832556  5381 consensus_meta.cc:287] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.004s
W20260709 13:14:35.833081  5501 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.990s	user 0.007s	sys 0.006s
I20260709 13:14:35.861781   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:35.863058   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:35.863914   997 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.836719  5597 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:35.837980  5613 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:14:35.839210  5574 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.822456  5622 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 13:14:35.844453  4993 consensus_meta.cc:287] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.784s	user 0.008s	sys 0.003s
I20260709 13:14:35.845471  5565 consensus_queue.cc:260] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.823413  5487 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.816268  5214 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:35.848928  1599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.017s	sys 0.021s
W20260709 13:14:35.912976  1599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.017s	sys 0.021s
W20260709 13:14:35.869737  5612 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:14:35.875161  5619 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:14:35.963793  5160 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.964617  5597 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:35.879086  5473 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.874930  5501 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.974004  5501 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:35.875380  5381 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:35.976927  5381 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.878700  4993 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:35.978024  5630 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:14:35.978879  4993 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:35.874624  5590 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.979571   854 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:35.964390  5160 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.844399  5113 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:35.990228   925 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:36.001206  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 148506us
W20260709 13:14:35.997563   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.011121   997 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:36.013617   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.029300   849 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.031903  5572 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:36.033592  5610 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.651s	user 0.001s	sys 0.010s
W20260709 13:14:36.000902  5588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.007s	sys 0.025s
W20260709 13:14:36.020895  5607 log.cc:927] Time spent T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.625s	user 0.005s	sys 0.017s
I20260709 13:14:36.025926   927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 2276) took 1214 ms. Trace:
W20260709 13:14:36.026141  5626 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.264s	user 0.003s	sys 0.000s
I20260709 13:14:36.037359   848 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:36.045002   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.049830   853 raft_consensus.cc:3060] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.046957   927 rpcz_store.cc:276] 0709 13:14:34.811425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.811574 (+   149us) service_pool.cc:224] Handling call
0709 13:14:34.811948 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:34.851660 (+ 39712us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:14:34.851688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:34.864579 (+ 12891us) log.cc:844] Serialized 52 byte log entry
0709 13:14:35.286167 (+421588us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:35.286204 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.286274 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:36.025673 (+739399us) raft_consensus.cc:1685] finished
0709 13:14:36.025692 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.025869 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36272,"raft.queue_time_us":39851,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":11544,"thread_start_us":379611,"threads_started":1,"wal-append.queue_time_us":658208}
I20260709 13:14:36.048754  5276 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:36.064042   849 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
W20260709 13:14:36.045672  5588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.007s	sys 0.025s
I20260709 13:14:36.069602   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:36.029699  5113 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.003443  5275 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.074606  5577 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:36.086156   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 721) took 1249 ms. Trace:
I20260709 13:14:36.086606  5617 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for earlier term 6. Current term is 7.
W20260709 13:14:36.088702  1567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.046s	sys 0.137s
I20260709 13:14:36.064703  5632 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.088119   923 rpcz_store.cc:276] 0709 13:14:34.837206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.841236 (+  4030us) service_pool.cc:224] Handling call
0709 13:14:36.085918 (+1244682us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b5400082854. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:36.086061 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639800576}
W20260709 13:14:36.090371  1567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.047s	sys 0.137s
W20260709 13:14:36.088791  5557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.004s	sys 0.012s
W20260709 13:14:36.100179   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 342287us
W20260709 13:14:36.101016  5557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.005s	sys 0.013s
I20260709 13:14:36.092723  5577 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:36.099952  5113 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:36.114454  1594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 484) took 1358 ms. Trace:
I20260709 13:14:36.115885   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:36.115720  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:36.118726   997 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 14: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:36.118754  1000 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.117595  1594 rpcz_store.cc:276] 0709 13:14:34.756338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.756451 (+   113us) service_pool.cc:224] Handling call
0709 13:14:34.756775 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:34.785656 (+ 28881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:34.785687 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:34.812952 (+ 27265us) log.cc:844] Serialized 26 byte log entry
0709 13:14:35.222288 (+409336us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:35.222327 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.222393 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:36.020240 (+797847us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:36.110343 (+ 90103us) raft_consensus.cc:1685] finished
0709 13:14:36.110362 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.114390 (+  4028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10676,"raft.queue_time_us":13376,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":3242,"spinlock_wait_cycles":100548992,"thread_start_us":367627,"threads_started":1,"wal-append.queue_time_us":544433}
I20260709 13:14:36.092612  5632 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.131678  5557 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.134443  5557 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.125737  5113 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.158354  5113 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.115520  5624 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:14:36.163909  1000 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:36.140214  5214 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.184237  5550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bbb1b886474a538a2f1a32dd91b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:36.186650  5541 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.193523  5473 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:36.194258  5473 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.195737  5575 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.196455  5575 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.199165  5575 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.148542  5633 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:36.200026   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 332125us
I20260709 13:14:36.189930   818 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:36.203053   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:36.204406   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.205282  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.206266   927 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.206502   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.206856   995 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:36.207716   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.210423   924 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:36.208952   921 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:36.207317  1599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45838 (request call id 487) took 1429 ms. Trace:
I20260709 13:14:36.211829   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.213423   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.214000   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.214020   927 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:36.214799  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.215384   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.216323   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.216514   925 raft_consensus.cc:1240] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.187209  5541 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.218031   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.211552  1599 rpcz_store.cc:276] 0709 13:14:34.778060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.778184 (+   124us) service_pool.cc:224] Handling call
0709 13:14:34.778450 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.778689 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:35.127415 (+348726us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.206067 (+1078652us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.207012 (+   945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.207056 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:36.207101 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.207170 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.207266 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15646,"prepare.queue_time_us":411018,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":10520,"raft.queue_time_us":15998,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":1287074,"threads_started":3,"wal-append.queue_time_us":983780}
I20260709 13:14:36.171566  5646 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.221751   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.222635   929 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.223546   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.224257   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.224875   930 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:36.221252   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.224813   927 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:36.226450   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.227200   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.227820   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.228294   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.227433   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.208989  5499 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:36.227795   927 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
W20260709 13:14:36.230296  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.208688  5617 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:36.231767   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.233610   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.244029   921 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:36.258481   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.251998   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.251192  5561 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:36.259765  5561 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:36.260197   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:36.260489  5634 log.cc:927] Time spent T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:14:36.260655  5561 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.261300   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.261704  5275 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:36.261847  5561 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:36.210597  5633 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:36.253039  5640 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:36.263309  5557 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.263363   924 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:36.224531  5646 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.233026  5374 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.264755   854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2568) took 1316 ms. Trace:
I20260709 13:14:36.264576  5374 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.265129   854 rpcz_store.cc:276] 0709 13:14:34.948747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.948873 (+   126us) service_pool.cc:224] Handling call
0709 13:14:34.949143 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.980026 (+1030883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.264533 (+284507us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540041f5d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:36.264700 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369305472}
I20260709 13:14:36.265753  5646 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.219802  5564 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.261309  5160 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.263319   922 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Committing config change with OpId 10.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:36.266682  5374 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.267663  5632 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.221624  5577 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.255422   846 raft_consensus.cc:2468] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:36.272353   921 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 14: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:36.229910  5643 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.273128  5643 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.239709  5487 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.281524   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.282136  5073 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:36.282351   928 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.282620  5073 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:36.266412  5564 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.252971 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:36.305830   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.306186   853 raft_consensus.cc:2468] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:36.256604   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.219154  5544 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:36.306464   924 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.308184   930 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.269062  5590 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.311990   847 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:36.320364   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.312484   927 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:36.326359   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.327700   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.329084   851 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.329329   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.330410   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.330756   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:36.331442   847 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:36.333416   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.336601   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:36.337580   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.342087  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:36.349920   924 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:36.358615   927 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:36.360154  1599 raft_consensus.cc:2368] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.324003  5617 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:36.364401   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.364413   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.328615  5643 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:36.395001   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.328370  5487 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.283669  5073 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.338727  5359 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:36.402472   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.404613   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.323415  5544 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.406046   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.411594   847 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:36.415974   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.420789   925 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:36.425814   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 47535.6us
I20260709 13:14:36.409507  5276 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.427305   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:36.404677  5588 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.424425   821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.3:44411 timed out after 1.923s (SENT) [suppressed 15 similar messages]
I20260709 13:14:36.405103  5597 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:36.401525  5575 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:14:36.410768  5374 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.439520   818 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:36.418468  5359 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.422619  5621 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:36.471863   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 94680.9us
I20260709 13:14:36.423080  5644 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:14:36.493335   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:36.494128   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.497112   968 leader_election.cc:400] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.498313   968 leader_election.cc:403] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:36.431114  5564 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.437770  4993 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:36.440303  5473 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:36.505695   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 60936.1us
I20260709 13:14:36.490998  5567 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:36.512207   818 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:36.509452  5611 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:14:36.438108  5276 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.441310  5588 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.518862   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.500932  5577 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:36.507182  4993 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:14:36.517868   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.520859  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.508363  5644 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:36.443106  5597 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.516680  5590 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.523399  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:36.524398   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:36.522091   998 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:36.519423  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:36.526078   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 101681us
I20260709 13:14:36.532500  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.560024  1000 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:36.561152  1000 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:36.562804  1000 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:36.564330  5575 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:36.565162  5575 consensus_queue.cc:1243] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:36.512328  5541 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.490252  5374 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.569514  5374 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.534008  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.523347   996 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:36.518841  5656 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.557250   999 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.570690  5656 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.573645  5656 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.495762  5621 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.493481  5617 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.510808  5564 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:14:36.519034  5567 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:36.577014   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 52055.7us
I20260709 13:14:36.580740   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.583431   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:36.584728  5597 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:36.525846  5359 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.595257   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:36.610065   928 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:36.529564  4993 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.616564   929 raft_consensus.cc:3060] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.532814  5267 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:36.620445   998 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:36.652313  5567 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.656498  5374 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.590430  5649 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:14:36.616436   999 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
W20260709 13:14:36.666936   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 107562us
I20260709 13:14:36.595022  5565 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:36.669449   967 leader_election.cc:400] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.594718  5588 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.589321  5557 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.672880   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:36.593042  5276 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.674661   929 raft_consensus.cc:2468] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:36.679817   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:36.532399  5577 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.688771   967 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:36.670362  5374 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.689666  5588 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:36.690104  5588 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.690477  5588 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.691387  5374 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.697082   967 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:36.673556  5663 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.707831   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:36.785089  1566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.665438  5590 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:14:36.795769  1003 raft_consensus.cc:2410] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:14:36.796360   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 88620.1us
I20260709 13:14:36.805056   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:36.695381  5501 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:36.806222   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 294054us
W20260709 13:14:36.795413  5660 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:14:36.820931  5595 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:36.696333  5667 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:36.681710  5557 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.795837  5544 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.792479  5657 raft_consensus.cc:2764] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:14:36.809713  5597 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:36.815948  5501 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:36.870645   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 355617us
I20260709 13:14:36.878690  5669 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.879371  5669 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.881057  5276 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.881673  5669 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.881588  5276 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.883716  5276 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.874477  5501 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:36.893006  5557 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:36.896914  5561 consensus_meta.cc:287] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.002s
I20260709 13:14:36.896660  5577 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.818849  5663 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.904029  5621 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.904281  5588 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.910805  1002 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
W20260709 13:14:36.911751   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 93919.7us
I20260709 13:14:36.870043  5359 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:36.896652  4993 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:36.872470  5267 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:36.917491   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:36.791293  5644 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.942937   967 leader_election.cc:400] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14.
I20260709 13:14:36.879308  5671 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:14:36.939950   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 124491us
I20260709 13:14:36.941120  5656 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:36.936448  5359 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:36.957176  5267 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.941294  5590 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:36.967985  5642 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.745s	user 0.000s	sys 0.005s
I20260709 13:14:36.957640  5567 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:36.972611  4993 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.971359  5501 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:36.973337  5663 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:36.974125  5633 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:36.984611  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.984807   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:37.029548  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:37.029726   846 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.990199  5573 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.030270  1599 raft_consensus.cc:2393] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:36.994982  1567 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:36.997702   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:36.998309   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.998452   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.031961   847 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:36.999020  5636 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.033672  5636 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.034579  5501 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:36.999963   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.035777  5636 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.036553  5280 raft_consensus.cc:2804] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:37.036751   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.037140  5280 raft_consensus.cc:697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:37.037415   853 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:14:37.038259   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.038076  5280 consensus_queue.cc:237] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.040020   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.040319  5359 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.042550  5561 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:37.043084  5561 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:37.000351  5670 log.cc:927] Time spent T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:37.000306   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.049851   855 raft_consensus.cc:2410] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:37.000684   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.004376   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.049913   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.003780  5575 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.007725   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:36.947185   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.011860   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.021543   895 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.040357  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.045997  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:37.008733  5671 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:37.011394  5588 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.011768  5563 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.039434   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.057596   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.043385  5567 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.060523  1003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:37.063925   846 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:37.063930  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.030166  5590 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.059522  5622 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:37.024407  5682 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:37.024698  5577 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:37.025876  5557 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.065434  5575 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.066219  5667 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:37.066769  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:37.066493  5563 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.067163   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.067508  5160 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:37.093209  1596 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:37.070935  5567 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.074031  5561 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.074666  1593 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:14:37.077140   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 65363.8us
I20260709 13:14:37.101847   821 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.077977  1002 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:37.078896   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.064029   895 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:37.076740   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 130033us
I20260709 13:14:37.103188   995 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:14:37.104737  1003 raft_consensus.cc:606] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.105242  1003 raft_consensus.cc:614] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:37.089126   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.106779   999 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:37.092433  1001 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.111212   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.112421   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.113651   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.115232   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.120903   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.128644   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.092238  5625 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:37.129187  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.134305   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.137369   930 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:37.097259  5374 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.087993  5679 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:37.097520  5676 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.101353  5657 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.147714   927 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:14:37.159060   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57296.5us
I20260709 13:14:37.159442  1000 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:14:37.150308   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 86391.1us
I20260709 13:14:37.153409   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.141782  5625 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.162254   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.173660  1001 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:14:37.174831   995 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:37.093470  5682 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.165288   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.144368  5275 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.175246   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:37.177953   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 166569us
I20260709 13:14:37.178355   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.100965  5577 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.179435   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.096215  5557 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.180070   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.176028  5622 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:37.090042  5671 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.184842   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.101480  5667 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.185803   998 raft_consensus.cc:2368] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:37.187963   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.191442   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.196965  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.177171  5625 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:37.202827   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.184119  5673 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:37.212105   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.214506  1564 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.203637  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.204457  1000 pending_rounds.cc:85] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 4
I20260709 13:14:37.222127  1000 pending_rounds.cc:107] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:14:37.222469  1000 raft_consensus.cc:2889] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:37.153223  5676 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.183446  5633 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.148392  5374 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.192751  5144 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.188179  5679 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:37.193212  5644 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:37.225162   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.184026  5677 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:14:37.226745   895 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.222922   896 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.230228   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.231107   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.209729  5622 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:37.228406  5302 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.215240  5669 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:37.201546  5663 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:14:37.158020  5657 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.243216  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.241220   969 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:37.241988   997 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:14:37.194391  5655 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:14:37.262566   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.230088  5113 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:37.231051  5499 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:37.241860  5692 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.284842   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:37.274516  5113 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:37.287125  5685 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:14:37.297529   997 raft_consensus.cc:2376] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11. Re-sending same reply.
W20260709 13:14:37.287698  5687 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:14:37.259333  5556 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:37.300973   997 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:37.292150  5684 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:14:37.292695  5575 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.294509  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.295023  5657 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.255019  5669 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.316390  5669 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:37.301769   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 122388us
I20260709 13:14:37.290477  5667 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.302721   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.277007  5644 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:37.282094  5656 raft_consensus.cc:2804] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:37.245499  5131 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:37.314384   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 161028us
I20260709 13:14:37.254110  5597 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:14:37.298040  5113 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:37.303990   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.318897   821 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:37.321182   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 74502.2us
I20260709 13:14:37.327744   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.301641  5622 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:37.329137   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.329255   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.311259  5556 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.318116  5563 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.332916  5374 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.333348  5577 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.336931   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.337601   930 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:37.330480   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.340765   927 raft_consensus.cc:2393] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:37.343242   896 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:37.343964   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.345685   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.346263   923 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:37.354182   926 raft_consensus.cc:2393] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:37.321527  5590 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.356431   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.358673   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.368151   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.328550  5575 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.365523   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.370143   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.374847   923 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:37.339052  5160 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:37.339144  5556 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.331465  5622 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:37.392218   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.295629  5679 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.339465  5574 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.355751  5563 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.355573  5677 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.395059   928 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:14:37.395941   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:37.400466   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 131054us
I20260709 13:14:37.408478  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:37.390451  5687 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:14:37.330281  5131 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.413620   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.414570   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.391449  5160 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:37.393863  5374 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.395241  5622 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:37.444615   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 208203us
I20260709 13:14:37.419021   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.376435  5644 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:14:37.437330  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.418634   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.459616   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:37.460482   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 49760.4us
I20260709 13:14:37.468729   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.470211   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:37.476171   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:37.479466  1596 raft_consensus.cc:2410] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:37.409711  5575 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.416875  5563 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.419343  5656 raft_consensus.cc:697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:37.439087  5702 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:14:37.482713   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 126573us
I20260709 13:14:37.438017  5557 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.494832  5557 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.444178  5655 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.462363  5160 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.498601   930 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:37.462597  5374 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.499892  5563 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.502028  5563 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:37.502446  5550 consensus_meta.cc:287] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:14:37.503877  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.504592  1594 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:37.504503  5656 consensus_queue.cc:237] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.442906  5696 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.474649  5678 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.412269  5677 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.477825  5682 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.496024  5655 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.508600   896 leader_election.cc:400] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:37.528846   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.551442   896 leader_election.cc:403] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:37.520865   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:37.553344   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44948.8us
I20260709 13:14:37.553664   850 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:37.556250   855 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:37.590104   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.590595   853 raft_consensus.cc:2393] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:37.590807   997 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:37.503909  5550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:37802 (request call id 750) took 3853 ms. Trace:
I20260709 13:14:37.590731  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:37.486860  5671 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.595429  5673 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:37.551362  5597 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:37.512760  5590 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:37.520800  5575 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.512538  5696 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.533293  5708 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.606441  5597 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.606827  5280 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:37.611145  5131 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.614665  5590 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:14:37.646795   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 189560us
W20260709 13:14:37.639767   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 163686us
I20260709 13:14:37.649123   969 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.660966   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.662070   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:37.662957   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 183419us
I20260709 13:14:37.637105  5597 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:37.630714  5280 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.615128  5701 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:37.673301   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.613226  5622 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.681761   926 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:37.673346   896 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.672228  5280 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:37.695053   926 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:37.695864   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 62463.8us
W20260709 13:14:37.692662  5699 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:14:37.622402  5708 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.735842   928 raft_consensus.cc:2410] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:37.742404  5696 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:37.744246  5713 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:14:37.746122  5073 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:37.746959  5625 consensus_meta.cc:287] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Time spent flushing consensus metadata: real 0.524s	user 0.003s	sys 0.004s
I20260709 13:14:37.748077  5726 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:37.748633  5726 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:37.720408  5704 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:14:37.749079  5726 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.701822  5590 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.757416   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.753631  5725 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.599555  5550 rpcz_store.cc:276] 0709 13:14:33.691406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.700125 (+  8719us) service_pool.cc:224] Handling call
0709 13:14:34.326259 (+626134us) ts_tablet_manager.cc:934] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
0709 13:14:36.102121 (+1775862us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:14:36.993669 (+891548us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:36.993789 (+   120us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:14:37.503024 (+509235us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:14:37.503376 (+   352us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:37.503734 (+   358us) ts_tablet_manager.cc:989] Replica registered.
0709 13:14:37.503797 (+    63us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":24213,"tablet-copy.queue_time_us":467511,"thread_start_us":133371,"threads_started":2}
W20260709 13:14:37.781533   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 73358.5us
W20260709 13:14:37.755117  5700 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:14:37.720989  5709 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.791204   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:37.795922   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.796208   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.783864  5696 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:37.797484  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.747375  5625 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.803810  5550 tablet_copy_client.cc:806] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:37.803624  5669 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:37.804679  5721 log.cc:927] Time spent T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:14:37.798213  5644 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:14:37.811743  1594 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:37.765027  5710 consensus_queue.cc:1059] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:14:37.800374  5556 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:37.822860  5711 log.cc:927] Time spent T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:14:37.823868  5160 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.824337  5718 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:14:37.824987   930 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:37.829105   925 raft_consensus.cc:2393] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:37.779083  5722 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:14:37.836966   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:37.801961  5696 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:37.809994  5655 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.794988  5113 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:37.844126   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 195076us
I20260709 13:14:37.809671  5575 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.845139   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:37.852635   999 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.847288   969 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:37.838824  5567 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.844465  5550 tablet_copy_client.cc:670] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:37.860208   969 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.843762  5696 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.851220   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.863739   927 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:37.868785   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.860230  5677 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.871678   999 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:37.865288   969 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.890455   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.865986  5702 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:37.877624  5723 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:37.889751  5710 raft_consensus.cc:2764] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:37.897359  5677 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.897688  5669 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.865285  5622 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.886613  5708 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:37.879437  5267 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:37.899189  5656 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:37.878505  5731 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.902274  5160 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.903802  5722 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:37.908901   924 raft_consensus.cc:2410] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:14:37.909787  5702 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.991117   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.864373  5567 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.919376   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.893994  5718 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:37.915292   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.929145  5722 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:38.117246   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:37.930421  5113 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.920768  5677 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:37.938007  5633 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:37.941272   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:37.945570   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:37.950214   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.232648   924 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:38.234783   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.226766   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.236280   855 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 6.
W20260709 13:14:38.237723   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 277368us
I20260709 13:14:38.238690   821 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.005467   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:37.997990  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.047673  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:37.964578  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.995410  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.896854   999 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:38.005468  5556 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:37.976207  5736 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:37.943499  5597 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.925709  5267 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:37.951956  5625 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:37.978951  5740 log.cc:927] Time spent T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.081s	user 0.002s	sys 0.000s
I20260709 13:14:38.056902   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:37.964529  5676 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.292773  5267 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.293484  5267 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:38.293825  5267 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:38.294230  5267 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.222836  5718 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.298301  5718 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.076190  5577 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:38.299029  5577 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:38.300050  5577 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.301275  5577 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.061561  5280 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.304219  1600 raft_consensus.cc:1240] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:14:38.305490  5280 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:37.990898  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.953347  5726 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.309849  5726 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:37.998888   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.086786  5643 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:38.089427  1003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34972 (request call id 29) took 1029 ms. Trace:
I20260709 13:14:38.310994  5643 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.311439   930 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:38.310971  1003 rpcz_store.cc:276] 0709 13:14:37.060256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.060387 (+   131us) service_pool.cc:224] Handling call
0709 13:14:38.089208 (+1028821us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54003e4254. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b39853 00007f0d07b0760a 00007f0d0108d7d4 00007f0d0108d762 00007f0d0108d691 00007f0d0108d60c 00007f0d0108c8c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:38.089339 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95856512}
I20260709 13:14:38.312608   927 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:38.313259  5643 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.314025   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.090510  5696 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.094825  5374 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.069398   995 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:38.316629  5696 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.100513  5322 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.100664   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.101593  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.056370  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:38.064070   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.318243   926 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:14:37.991693  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:37.962707  1594 raft_consensus.cc:3060] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.951278  1002 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:37.969691  5714 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:14:37.959014  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.100585   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.107028   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.107029   850 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:38.109157   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.321967   848 raft_consensus.cc:2368] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:38.322693   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.323345  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:38.111469   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.117422   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:37.918694  5723 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.217511   854 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:38.218971   851 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:38.219626   852 raft_consensus.cc:1240] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:38.159166   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:38.220597   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 305340us
I20260709 13:14:38.250515  1000 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:38.251660  1598 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 5.
I20260709 13:14:38.248620   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.245817   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.239478  5302 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:38.222554  5633 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:38.251328  5669 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:38.266086  5501 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:38.238371  4993 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.260288   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:38.349246   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 89135.1us
W20260709 13:14:38.259232   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 253862us
I20260709 13:14:38.351397   821 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:38.253400  5671 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.360100  5671 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:38.362397  5696 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.363656  5597 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.363823  4993 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.364423   927 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:38.365681  5677 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.365814  4993 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.347533  5692 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:14:38.280982  1001 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 16: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:38.291675   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 234860us
I20260709 13:14:38.321414   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.323431   853 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:38.110775   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.338461   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.368408   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:38.371122   999 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:38.369064   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.340763   849 raft_consensus.cc:1240] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:38.374063  1000 raft_consensus.cc:2468] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:38.374595   849 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:38.375988  1002 raft_consensus.cc:1240] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:14:38.339259  1564 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.341087   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.318148  5657 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.347117  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.353818   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.341504  5302 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.385015   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:38.385252  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46058.5us
I20260709 13:14:38.354604  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:38.355654   847 raft_consensus.cc:1275] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:38.356125   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.356706  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.357837  5643 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:14:38.371174   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.394328   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.394901   846 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:38.340689   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.377646   995 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:14:38.377631  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.377992   850 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:38.378340   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.379962   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.383109   851 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:38.383502  1000 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:38.392357   848 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:38.396957   894 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:38.392642   855 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:38.399832   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:38.317502  5322 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.396402  5692 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:38.409335   896 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:38.412616   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44297.2us
I20260709 13:14:38.413189   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:38.418586  1600 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:38.383574  5656 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.424273  5302 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.424494   818 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.424973  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.426156  5625 consensus_queue.cc:1059] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:38.426281  5668 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:38.426656  5501 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.427198  5501 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.429184  5655 raft_consensus.cc:3055] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:38.429430  5501 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.429657  5655 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:38.422224  5679 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:38.337040  5722 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.427400  5113 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:38.422070  1594 raft_consensus.cc:1275] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:38.432788  5707 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:38.420323   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:38.342924  5275 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:14:38.440083   996 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:38.440677   996 raft_consensus.cc:606] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.435405   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.368353  5726 consensus_queue.cc:1059] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:14:38.442934   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.445186  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.405943  5708 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:38.447369   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.445446  5597 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:38.457458  5622 raft_consensus.cc:2804] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:38.411674  5692 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:38.464022   853 raft_consensus.cc:2468] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:38.457123  5561 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:38.436681  5732 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.489939   922 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:38.469051   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:38.472844  1566 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.471386   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.493782   851 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:38.495232   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.497422   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.499110   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.501055   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.442762  5275 consensus_queue.cc:1243] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:38.441442  5113 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:38.457702  5683 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.512269   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:38.508476   855 raft_consensus.cc:2368] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:38.427350  5625 consensus_queue.cc:1059] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:14:38.450456  5708 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.511337  5622 raft_consensus.cc:697] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:38.487756   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:38.464962  5692 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:38.489631  5593 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.528570   895 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.439074  5656 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.527663   851 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:14:38.532128   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.535655  5561 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:38.535670  5144 consensus_queue.cc:1059] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:14:38.538280  5556 raft_consensus.cc:3055] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:38.538604  5556 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:38.541221   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:38.541889   998 raft_consensus.cc:606] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.542271   998 raft_consensus.cc:614] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:38.420547  5633 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.539281  5556 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.524927  5572 raft_consensus.cc:2804] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:38.525846  5708 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:38.521009  5113 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:38.571185  5113 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.560570   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.552896  5565 consensus_queue.cc:1048] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:14:38.579173  5741 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:38.562711   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.562901   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.565539  5556 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:38.581482   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.565608  5567 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.566393  5656 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.567001  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:38.585268  1002 raft_consensus.cc:606] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.567746  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.548216  5701 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:38.569342  5267 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.520854  5683 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.541918  5678 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:38.521260  5473 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:14:38.569432  5322 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.561858  5677 raft_consensus.cc:3055] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:38.587986  5501 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:38.593421  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 233116us
I20260709 13:14:38.542470  5276 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:14:38.587538  5572 raft_consensus.cc:697] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:38.584048  5593 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:38.546434  5644 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:14:38.549340  5557 consensus_queue.cc:1059] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:14:38.599145   853 raft_consensus.cc:1240] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:38.599627   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:38.590866  5709 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:14:38.606400   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.591390  5678 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:38.595037  5677 raft_consensus.cc:740] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:38.602744  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.607198  5678 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.616492  1567 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.623570  5593 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:38.623943  5691 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:38.625571  5572 consensus_queue.cc:237] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.628868  5683 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.631203  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:38.631837  1002 raft_consensus.cc:606] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.632273  1002 raft_consensus.cc:614] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:38.613345  5280 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.606033  5622 consensus_queue.cc:237] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.640076 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:14:38.617211  5302 consensus_queue.cc:1175] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3bb5726887244caca146e176cc6bd070" responder_term: 10 status { last_received { term: 10 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 10 index: 7 } }
I20260709 13:14:38.622480  5667 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:38.674819  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 59407.3us
I20260709 13:14:38.676381  1567 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.646169  5667 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:38.681720  5501 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:38.689275  5322 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:38.689888  5322 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:38.696171 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:38.696907   749 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 13, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 13 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:38.698992  5683 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:38.699115   749 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:38.699565  5683 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.701754  5683 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:38.702601  5678 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.707674   851 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:38.641454  5677 consensus_queue.cc:260] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.710494  5667 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:38.676491  5267 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:38.713776   922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 738) took 1080 ms. Trace:
I20260709 13:14:38.721910   922 rpcz_store.cc:276] 0709 13:14:37.633061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.633223 (+   162us) service_pool.cc:224] Handling call
0709 13:14:37.649231 (+ 16008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.490416 (+841185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.713520 (+223104us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b5400300554. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:38.713719 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1932974976}
I20260709 13:14:38.723927  5677 raft_consensus.cc:3060] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:38.725814   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 299976us
I20260709 13:14:38.729220  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.730173  1001 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
W20260709 13:14:38.735208  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 58924.2us
I20260709 13:14:38.742251   849 raft_consensus.cc:1240] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:38.744520   969 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:38.749123   925 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:14:38.755081   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 148937us
I20260709 13:14:38.756326   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.758939   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.759837   929 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:38.762099   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.762699   925 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:38.763993   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.764261  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.752565  1001 pending_rounds.cc:85] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 5
I20260709 13:14:38.766289  1001 pending_rounds.cc:107] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:38.766618  1001 raft_consensus.cc:2889] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:38.737934  5730 raft_consensus.cc:3055] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:38.771209   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.691006  5322 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.764989  5709 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:38.758955  5280 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:38.785548   922 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:38.811046   921 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:14:38.815150  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 79246.6us
I20260709 13:14:38.816538  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.772758  5730 raft_consensus.cc:740] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:14:38.820346   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.767699  5676 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:14:38.767915  5501 consensus_queue.cc:1059] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:14:38.858608  5501 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:38.859254  5501 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.815881  5692 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.861342  5501 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.861764  5692 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.861999  5501 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:38.862363  5501 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:38.863149  5501 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.863942   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.864447  5676 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.864729  5501 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.866851   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:38.868655   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.870298   850 raft_consensus.cc:2393] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 6: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:38.873690   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 117770us
I20260709 13:14:38.901537   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.902256   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.836762  5572 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:14:38.864950  5676 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:38.866942  5668 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:14:38.911767   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 146607us
I20260709 13:14:38.828624  5759 consensus_queue.cc:1059] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:14:38.920825   923 raft_consensus.cc:1240] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:14:38.924803   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.899049  5621 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.922896   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:38.926031   850 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:38.921136   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.927100   929 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:38.928118   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
W20260709 13:14:38.928864  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 101816us
I20260709 13:14:38.910033  5678 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:38.913458  5595 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.934186   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:38.931578   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:38.931317   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.939177   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.918210  5709 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:38.902537  5703 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:14:38.956440   925 raft_consensus.cc:1275] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:38.924784  5572 consensus_queue.cc:1243] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:38.957973   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:38.958434   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:38.957562  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:38.960672   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:38.969520   929 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:14:38.977342   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:38.978976  1000 raft_consensus.cc:1275] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:38.982203  5678 consensus_queue.cc:1048] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:38.975876  4993 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 13:14:38.984073  5767 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:14:38.986418  5764 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:14:38.940208  5622 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:38.986970  5622 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:38.987360  5622 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.976939  5374 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.959357  5692 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:14:38.958238  5267 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.022780  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:38.902343  5730 consensus_queue.cc:260] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:38.966202  5769 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:14:39.034682   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58929.5us
I20260709 13:14:38.976398  5113 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.037876   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.013230  5144 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.016839  5550 tablet_copy_client.cc:538] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:39.041733  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:39.021149  5267 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:39.060876   928 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:39.036615  5769 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:39.042189  5669 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:39.046839  5771 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.013134  5778 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:14:39.038587  5567 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:14:39.053421  5730 raft_consensus.cc:3060] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:39.052115  5267 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.047485  5144 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.094980   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.098554  5144 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:39.091094  4993 consensus_queue.cc:1059] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:14:39.078904  5669 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:39.101239  5669 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.102487  5669 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:39.102476   996 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:39.084584  5753 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:39.117336   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:39.128772  5769 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:39.130595  5769 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:39.111291  5722 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:14:39.116089  5676 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:39.109269  5668 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.086537  5771 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.075304  5709 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:39.169808  5777 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:14:39.173628  5779 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:14:39.178495  5655 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.155018  5676 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:39.192873   923 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:39.190402  5144 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.197870  5144 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.198745  5771 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:39.199851  5144 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:39.201375  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:39.201407   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.202592  5588 raft_consensus.cc:993] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:39.203095  5588 raft_consensus.cc:1081] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
W20260709 13:14:39.205153  5708 consensus_meta.cc:287] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.005s
I20260709 13:14:39.205554  5708 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:39.206880  5747 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.678s	user 0.003s	sys 0.000s
I20260709 13:14:39.209715   847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:39.210456   847 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.210908   847 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:39.212241   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:39.145162  5668 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.214741   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:39.215533   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.224663   997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51128 (request call id 2254) took 1414 ms. Trace:
I20260709 13:14:39.155797  5667 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:39.228746   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.227449   997 rpcz_store.cc:276] 0709 13:14:37.798727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.981601 (+182874us) service_pool.cc:224] Handling call
0709 13:14:37.981911 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.982295 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.982323 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:38.233359 (+251036us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b2c000e6738. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d01438a11 00007f0d01448372 00007f0d02b7aa01 00007f0d02b3f43a 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2
0709 13:14:38.233832 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:14:38.389169 (+155337us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:38.389211 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.389269 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.206334 (+817065us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:39.212812 (+  6478us) raft_consensus.cc:1685] finished
0709 13:14:39.212839 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.212968 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":119459840,"thread_start_us":155013,"threads_started":1,"wal-append.queue_time_us":256337}
I20260709 13:14:39.237845   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.244225   967 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.225479  5655 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:39.208089  5567 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:14:39.257292   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.240037  5577 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:39.208292  5676 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.263070   847 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.265337   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.248453  5561 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:39.267076  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 221228us
W20260709 13:14:39.253031  5784 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:14:39.272914  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:39.254328  5708 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:39.289268  5792 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:14:39.242745  5772 consensus_queue.cc:1048] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:14:39.295137  5676 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:39.298656  5550 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:39.267956  5771 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.353361  5723 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:39.305419  5668 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:39.266788  5657 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:39.320175  5577 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:39.355062  5621 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:39.355456  5621 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:39.323984  5762 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:39.325826   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:39.281085  5691 consensus_queue.cc:1048] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:14:39.295214  5770 log.cc:927] Time spent T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
I20260709 13:14:39.252254  5791 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:39.344372  1002 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:39.345990   968 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.368912   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:39.347362   847 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:39.357803   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:39.359404  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:39.362396  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:39.371160   895 leader_election.cc:400] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:39.371227  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:39.372059  1000 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:39.372400  1000 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:39.362792   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:39.365108   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 119266us
I20260709 13:14:39.366532   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:39.374563   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:39.367380  1004 raft_consensus.cc:1240] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:39.367987  1003 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:39.376638   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:39.369328   995 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:39.379660   855 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:39.382589   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.384742   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.387732   821 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:39.389648   821 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:39.368146  5787 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:14:39.394215   855 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:39.381492  1000 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.396656  1000 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:39.367784  5722 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:39.402120   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 120011us
I20260709 13:14:39.394073  5113 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:39.403031   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:39.356328  5621 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.425508  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:39.410127   821 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.429833  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:39.434783  5682 raft_consensus.cc:993] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:39.430085   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 741) took 1473 ms. Trace:
I20260709 13:14:39.435807   924 rpcz_store.cc:276] 0709 13:14:37.956810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.256122 (+299312us) service_pool.cc:224] Handling call
0709 13:14:38.256360 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.393449 (+137089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.393479 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:38.393523 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:38.393551 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.429864 (+1036313us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400319a54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:39.429912 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.430032 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314859264}
W20260709 13:14:39.436260  5758 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.617s	user 0.000s	sys 0.002s
I20260709 13:14:39.404844  5577 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.444115   969 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:39.447088   849 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:39.447207  5657 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:39.435392  5595 raft_consensus.cc:1081] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:39.445123   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.413383  1002 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:39.444818  5633 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.391671  5669 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:39.460990   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 50956.2us
I20260709 13:14:39.462117   821 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.463449   821 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.390353  5679 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.399647  5667 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:39.455298  5787 log.cc:927] Time spent T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:14:39.469789  5692 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:39.426019  4993 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:14:39.451499  5769 consensus_queue.cc:1059] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:14:39.409505  5752 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.398944  5567 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:14:39.489712  5671 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.383616  5791 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.456741  5657 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:39.404381  5683 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.492762   821 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.495293  1000 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
I20260709 13:14:39.469026   997 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:39.468338  5113 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.400626  5625 consensus_queue.cc:1059] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:14:39.464357  5696 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:39.504290  5748 log.cc:927] Time spent T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.915s	user 0.001s	sys 0.003s
I20260709 13:14:39.530469  5791 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:39.510489  5575 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:39.487830  5668 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:39.531945  5668 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.486140  5621 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:39.536911   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.538033  5280 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:39.538640  5280 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:39.539858  5617 raft_consensus.cc:3055] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:39.540409  5617 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:39.541531  5617 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.543118   930 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:39.505788  5789 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:14:39.544548  5789 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:39.544934  5789 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:39.513155   855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:39.546861   855 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.507434  5622 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.547554   855 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.546852   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:39.534886  5668 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:39.557996   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:39.495932  5267 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.559037  5575 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.560832  5669 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.556874  5617 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:39.588864   851 raft_consensus.cc:1240] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:39.545931  5789 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.587460  5575 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:14:39.604892  5280 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.607437  5280 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:39.608099  5789 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:39.617336  5267 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:39.623114  5759 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:39.626869  5759 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:39.628389  5759 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.624430   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:39.632912   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:39.651361   893 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.663825   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.664842   850 raft_consensus.cc:2393] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 16: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:39.665656   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:39.666215   850 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:39.668421  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:39.639937   855 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.671784   846 raft_consensus.cc:1240] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:39.623162  5761 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.016s	sys 0.033s
I20260709 13:14:39.582760  5113 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:14:39.676155   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:39.677338   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 76590.5us
I20260709 13:14:39.676914   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:39.667526  5796 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:14:39.721412   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:39.730022   850 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:39.671423  5669 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:39.786820   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.712275  5622 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:39.695825  5761 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.017s	sys 0.034s
I20260709 13:14:39.668663  5575 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:39.674249  5779 log.cc:927] Time spent T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:14:39.810482  5768 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:39.813314   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:39.818192   847 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:39.846966  5561 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.819905   849 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:39.697860  5759 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:39.821825   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:39.874543   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 52801.9us
I20260709 13:14:39.871477  5561 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.880426  5561 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:39.808898  5669 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:39.892570  5679 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:39.874260  5768 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.887395  5575 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:39.902941  5679 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.911887  5621 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:39.915096  5805 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:39.926620  5595 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:39.948764  5595 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.931145  5679 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:39.943949  5805 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.945570  5669 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:39.972955  5805 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:39.951507  5595 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:39.984335   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.942698  5621 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:39.995026  5621 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:39.995595   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.992626  5696 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:14:40.057077  5816 log.cc:927] Time spent T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:14:40.065488  5820 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:14:40.052170  5575 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.062829  5696 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.075189  5768 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.108718  5696 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.112077   855 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.119649   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46360 (request call id 2636) took 1114 ms. Trace:
I20260709 13:14:40.120103   851 rpcz_store.cc:276] 0709 13:14:39.005640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.005769 (+   129us) service_pool.cc:224] Handling call
0709 13:14:39.020290 (+ 14521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.589599 (+569309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.119368 (+529769us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54003b1ad4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b3fe01 00007f0d02b3e83e 00007f0d07b03419 00007f0d0107abd4 00007f0d0107ab62 00007f0d0107aa91 00007f0d0107aa0c 00007f0d01079cc2 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:40.119568 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1306899712}
I20260709 13:14:40.120779  5679 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.123416  5322 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.124734  5567 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.125357   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.125386  5567 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.128768  5567 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.131700  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:40.141942   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.148295  5774 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.148896  5774 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:40.148468  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.157841   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:40.162701  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.165853   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.163292  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:39.697572 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:40.166532   851 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:40.167853  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:40.151295  5774 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.175385  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:40.173601  5621 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.177096  1592 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:40.176988  5621 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.179152  5621 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.179986   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.172988  5814 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:40.173822  5768 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:40.181792  5768 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.182190  5768 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.174041  5679 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.176047  1599 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:40.188247  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:40.188750  1599 raft_consensus.cc:3060] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:40.195879  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:40.196226  1598 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:40.198930  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:40.199440  1598 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:40.203989  5683 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:40.213331  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:40.215445   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.216246   846 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:40.243984   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.245527   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.206240  5683 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:40.255985   854 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:40.260749   753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 228) took 15469 ms (client timeout 15000 ms). Trace:
I20260709 13:14:40.263289   928 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:40.264046   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.264178   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.264664   930 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:40.219748  5679 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.272435   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.276271   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.247493  5267 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.273481  5643 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:40.288892  1598 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 8.
W20260709 13:14:40.261163   753 rpcz_store.cc:269] 0709 13:14:24.791452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.851530 (+ 60078us) service_pool.cc:224] Handling call
0709 13:14:40.260659 (+15409129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:35.553013 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:35.553733 (+   720us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:35.553791 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:14:35.553807 (+    16us) tablet.cc:662] Decoding operations
0709 13:14:35.554761 (+   954us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:35.554867 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:35.554899 (+    32us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:14:35.584886 (+ 29987us) tablet.cc:701] Row locks acquired
0709 13:14:35.587567 (+  2681us) write_op.cc:260] PREPARE: finished
0709 13:14:35.592818 (+  5251us) write_op.cc:270] Start()
0709 13:14:35.594477 (+  1659us) write_op.cc:276] Timestamp: P: 1783602875587825 usec, L: 0
0709 13:14:35.594538 (+    61us) op_driver.cc:348] REPLICATION: starting
0709 13:14:35.606380 (+ 11842us) log.cc:844] Serialized 3209 byte log entry
0709 13:14:36.472735 (+866355us) op_driver.cc:464] REPLICATION: finished
0709 13:14:36.538976 (+ 66241us) write_op.cc:301] APPLY: starting
0709 13:14:36.539103 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:36.549194 (+ 10091us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:36.549332 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:36.646181 (+ 96849us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:36.646543 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:14:36.646597 (+    54us) write_op.cc:312] APPLY: finished
0709 13:14:36.647053 (+   456us) log.cc:844] Serialized 81 byte log entry
0709 13:14:36.647287 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:36.670481 (+ 23194us) write_op.cc:454] Released schema lock
0709 13:14:36.678833 (+  8352us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:36.820430 (+141597us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:36.821709 (+  1279us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1596167,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":65912,"apply.run_cpu_time_us":14683,"apply.run_wall_time_us":250774,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37567,"num_ops":8,"prepare.queue_time_us":557806,"prepare.run_cpu_time_us":96096,"prepare.run_wall_time_us":538560,"raft.queue_time_us":288130,"raft.run_cpu_time_us":21637,"raft.run_wall_time_us":44087,"replication_time_us":877985,"spinlock_wait_cycles":2816,"thread_start_us":1052655,"threads_started":4,"wal-append.queue_time_us":512957,"wal-append.run_cpu_time_us":52782,"wal-append.run_wall_time_us":1534136}]]}
I20260709 13:14:40.245096  5778 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:40.271276  5565 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:40.311798  5565 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.271112  5473 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.313277   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:40.313390  5473 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.313864  5565 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.314035  5622 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:40.314364  5565 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:40.314917   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.314877  5565 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.302570  5561 consensus_queue.cc:1059] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:14:40.282294  5144 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:40.305753  5683 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.306797  5577 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.356285  5759 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:40.297741  5823 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.310249  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.356470  5144 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.356714  5577 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.357359  5823 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.358793  5144 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.359148  5577 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.315523   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:40.315917  5302 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:40.316237  5760 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.316149   895 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:40.271207  5723 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:40.282428  5708 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:40.280474  5805 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.319820   930 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:40.359417  5823 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.368407  5723 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.369313  5565 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.369509   995 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:40.356699  5683 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:40.361792  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:40.370484  5723 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.370842  1592 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 15: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:40.371124  5708 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:40.322638  5828 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.097s	user 0.003s	sys 0.000s
I20260709 13:14:40.299780  5643 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.373042  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.373534  5708 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.274271  5679 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.374442  5643 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.315474  5473 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:40.375223   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 59147.1us
I20260709 13:14:40.377741   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:40.378950   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.330947  5768 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.382088   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.345079  1599 raft_consensus.cc:2468] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 7.
I20260709 13:14:40.308192  5595 consensus_queue.cc:1059] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:40.311223  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.369336  5760 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.356936  5759 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:40.291823  5267 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.330459  1002 raft_consensus.cc:1240] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:40.410369   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.385488   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.386847   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.206763  5575 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:14:40.400259   995 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:40.403352   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.371411  5805 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:40.410969   852 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:40.414477   749 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 121) took 14734 ms (client timeout 15000 ms). Trace:
I20260709 13:14:40.359797  5577 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.427397  5768 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.430138  5805 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.436295  5577 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.438781   997 raft_consensus.cc:3055] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:40.439164   997 raft_consensus.cc:740] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:40.439926   997 consensus_queue.cc:260] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.441160   997 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.400211  5302 raft_consensus.cc:2804] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:40.446431   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:40.447146   846 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.409632  5657 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:40.459401   997 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:40.451730  4993 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.412947  5753 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.462329   855 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:40.408707  5682 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:40.465943   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45498.6us
I20260709 13:14:40.463771   895 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:40.468760   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.400384  5804 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:14:40.472294   846 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:40.472379   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.474354   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:40.475438   923 raft_consensus.cc:1240] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:40.408339  5473 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:14:40.476501  1566 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:40.478504   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.479842   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.480235   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.481475  5768 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.485605   921 raft_consensus.cc:1217] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Deduplicated request from leader. Original: 12.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 13:14:40.489629   926 raft_consensus.cc:1275] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:14:40.490022   926 pending_rounds.cc:85] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 7
I20260709 13:14:40.490291   926 pending_rounds.cc:107] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:14:40.490546   926 raft_consensus.cc:2889] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:40.490845   821 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:40.491533  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:40.492175  1598 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:40.492691   821 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:40.462888  5657 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.493780   921 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:14:40.414871   749 rpcz_store.cc:269] 0709 13:14:25.680461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.680686 (+   225us) service_pool.cc:224] Handling call
0709 13:14:40.414405 (+14733719us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:38.927298 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:39.083297 (+155999us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:39.085352 (+  2055us) write_op.cc:435] Acquired schema lock
0709 13:14:39.086439 (+  1087us) tablet.cc:662] Decoding operations
0709 13:14:39.144978 (+ 58539us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:39.155652 (+ 10674us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:39.157373 (+  1721us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:14:39.196700 (+ 39327us) tablet.cc:701] Row locks acquired
0709 13:14:39.196724 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:14:39.203180 (+  6456us) write_op.cc:270] Start()
0709 13:14:39.203288 (+   108us) write_op.cc:276] Timestamp: P: 1783602879203134 usec, L: 0
0709 13:14:39.203314 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:14:39.204025 (+   711us) log.cc:844] Serialized 913 byte log entry
0709 13:14:39.817076 (+613051us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b18002972e0. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa523f3 00007f0cffa547c1 00007f0cffa54779 00007f0cffa54709 00007f0cffa546d1 00007f0cffa539cd 000000000038f354 000000000038f199 00007f0cffa3583a
0709 13:14:39.817401 (+   325us) op_driver.cc:464] REPLICATION: finished
0709 13:14:40.075333 (+257932us) write_op.cc:301] APPLY: starting
0709 13:14:40.089758 (+ 14425us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:40.104467 (+ 14709us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:40.104501 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:40.105970 (+  1469us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:40.106175 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:14:40.106233 (+    58us) write_op.cc:312] APPLY: finished
0709 13:14:40.106475 (+   242us) log.cc:844] Serialized 33 byte log entry
0709 13:14:40.106668 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:40.106775 (+   107us) write_op.cc:454] Released schema lock
0709 13:14:40.106818 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:40.107230 (+   412us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:40.107257 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":860955,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":254368,"apply.run_cpu_time_us":4800,"apply.run_wall_time_us":33453,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12487,"num_ops":2,"prepare.queue_time_us":199459,"prepare.run_cpu_time_us":15879,"prepare.run_wall_time_us":354745,"raft.queue_time_us":83911,"raft.run_cpu_time_us":9714,"raft.run_wall_time_us":163140,"replication_time_us":613961,"spinlock_wait_cycles":2964736,"thread_start_us":379290,"threads_started":4,"wal-append.queue_time_us":181978}]]}
I20260709 13:14:40.498039   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.484597   924 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 7.
I20260709 13:14:40.494817   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:40.501982  5683 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:40.495844   750 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.502585  5683 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:40.503216   750 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:40.503502  5683 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.504714   750 catalog_manager.cc:5697] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:40.493925  5667 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:40.470523  4993 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.509999  4993 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.511128  5656 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:40.511533  5656 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.511726  1598 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:40.511929  5656 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:40.481856  5302 raft_consensus.cc:697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:40.506736   750 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 10, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.516803   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:40.518704   750 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:40.520020   851 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:40.494117  5790 raft_consensus.cc:2804] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:40.521898  4993 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:40.482306  5561 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:40.487399  5565 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:40.533269   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.533483   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.494580  5823 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:40.487028  5622 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:40.533849   924 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.522260  5790 raft_consensus.cc:697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:40.520771   750 catalog_manager.cc:5697] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:40.514490  5643 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:40.515234  5657 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.542856   750 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:40.588984   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.588714   750 catalog_manager.cc:5697] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:40.476882  5682 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.486920  5768 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.530786  5561 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.587822   925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42382 (request call id 2343) took 1120 ms. Trace:
W20260709 13:14:40.621062   818 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.542119  5565 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.634845   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.635571   851 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:14:40.522343  5302 consensus_queue.cc:237] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.476691  5753 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:40.637007   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:40.482450  5759 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.524909  5723 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:14:40.539891  5778 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:40.528398  5322 consensus_queue.cc:1059] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:14:40.653146  5832 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:40.587031  5643 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.621513   925 rpcz_store.cc:276] 0709 13:14:39.468362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.468520 (+   158us) service_pool.cc:224] Handling call
0709 13:14:39.468803 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.468981 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:39.503841 (+ 34860us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.270020 (+766179us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.587408 (+317388us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.587467 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:40.587516 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.587595 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.587759 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":770138,"prepare.queue_time_us":248650,"prepare.run_cpu_time_us":194,"prepare.run_wall_time_us":201,"raft.queue_time_us":739249,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":8982,"thread_start_us":345245,"threads_started":2}
I20260709 13:14:40.664785   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.650826   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.636857  5561 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:40.673414   853 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:40.674074   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.675166   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.676152   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.592491  5823 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:40.654446   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:40.646552  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.657414  5682 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.659399   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.681298   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.682081   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.657941  5768 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.683573   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:40.686173  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.659438  5280 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.672514  5643 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.592780  5622 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:14:40.686080  5831 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:14:40.664283  5832 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:40.711674   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.714372   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:40.715695  5730 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.716251  5730 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.717041  5832 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.718477  5730 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.719048  5730 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:40.719975   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.720129   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.720536   930 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:40.700156  5280 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.719515  5730 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.724834  5280 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.681203  5595 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.726543  5595 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.733398   930 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:40.698658   924 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:40.590610   750 catalog_manager.cc:5697] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.700399   894 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:40.703845   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:40.704025   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 209263us
I20260709 13:14:40.597039  5833 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.680735  5759 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.707384   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:40.618042  5790 consensus_queue.cc:237] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.697218  5718 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:40.680101  5473 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:40.744899   925 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:40.795652   750 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 14 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:40.799340   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.744623   926 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:40.840431   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.840610   997 raft_consensus.cc:1275] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:40.841368   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:40.738826   924 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.701529  5565 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.743498   818 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:40.688933  5656 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:40.736136  5556 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:40.872335   750 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:40.875653   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 132197us
I20260709 13:14:40.876873   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:40.879036   924 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:40.743913  5805 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:40.757989  5667 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:40.874402  5113 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.952049  5113 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.952639  5760 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:40.933120  5679 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.954239  5113 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.942833  5144 raft_consensus.cc:3055] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:40.958779   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.959059  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.959805   995 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:40.960561  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.961051   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.962018   995 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:40.962262  5835 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:14:40.962457   998 raft_consensus.cc:1240] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:40.959864  1002 raft_consensus.cc:2393] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:40.951323  5577 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:40.951233  5267 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:40.964918  5144 raft_consensus.cc:740] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:40.883481   750 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 7, term changed from 10 to 12, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:40.973469  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:40.980839  1592 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:40.994557  1003 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:40.746953  5833 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.006094   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.978502   750 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.885784  5669 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.007417   750 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.011176   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:40.885994  5683 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.026371  5561 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:40.978605  5557 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:41.027688  5557 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.028833  5834 consensus_queue.cc:1048] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:41.029487  5633 consensus_queue.cc:1059] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:40.937475  5656 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.030570  5633 raft_consensus.cc:3055] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:41.030932  5633 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:41.031783   996 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
W20260709 13:14:41.005602  5831 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:14:41.035183  5837 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:14:41.037545  5657 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.039783  5657 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.026113  5144 consensus_queue.cc:260] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.041112  5144 raft_consensus.cc:3060] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.871905  5723 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 13:14:41.029788  5557 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.015159   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.048449  5622 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.049201  5622 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.049237  5144 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:41.050582  5595 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.050354  5633 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.053287  5633 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.059933  5730 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.060351  5561 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.010008   750 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 12, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.062609   750 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:41.064002  5753 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.064348   750 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.067979  5550 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=1 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:41.069278  5550 tablet_bootstrap.cc:492] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:41.070197  5550 ts_tablet_manager.cc:1403] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 1.772s	user 0.041s	sys 0.018s
I20260709 13:14:40.974835  5805 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.064152  5622 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:41.076607  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.081892  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.084985  1002 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:41.079504  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.066529   750 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.100558  1001 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:14:41.015897  5669 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:41.113657   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:14:41.111310   969 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.006s	sys 0.095s
W20260709 13:14:41.137367   969 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.007s	sys 0.095s
I20260709 13:14:41.142846   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.143540   855 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
W20260709 13:14:41.149856  1598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.037s	sys 0.015s
I20260709 13:14:40.880869  5823 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.150239  1598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.037s	sys 0.015s
I20260709 13:14:41.083809  5622 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:41.158946   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.139299  5669 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.164412  5557 consensus_queue.cc:1059] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:41.178428  5557 raft_consensus.cc:3055] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:41.178906  5557 raft_consensus.cc:740] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:41.179912  5557 consensus_queue.cc:260] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.181414  5557 raft_consensus.cc:3060] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:41.173200  1592 raft_consensus.cc:2468] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:14:41.149973  5643 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.102979  5753 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.141408  5473 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.185648   968 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:41.215867   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:41.227049   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 71531us
W20260709 13:14:41.235886   927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.037s
W20260709 13:14:41.236428   927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.007s	sys 0.037s
I20260709 13:14:41.221922  5775 raft_consensus.cc:2804] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:41.259431  5669 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.260519  5550 raft_consensus.cc:359] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:41.261518  5550 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:41.261986  5669 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.262253  5550 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:41.211439  5473 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.252548   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:41.270478   847 raft_consensus.cc:2393] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:41.257661   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:41.219183  5753 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:41.291762   894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.023s	sys 0.011s
W20260709 13:14:41.292207   894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.023s	sys 0.011s
I20260709 13:14:41.239949  5683 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.275511  5473 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:41.292984   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 592607us
I20260709 13:14:41.267714  5501 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:41.274896  5775 raft_consensus.cc:697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:41.306342   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:14:41.325877  1599 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.027s	sys 0.031s
W20260709 13:14:41.326329  1599 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.027s	sys 0.031s
I20260709 13:14:41.293586   848 raft_consensus.cc:2393] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
W20260709 13:14:41.299326  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 436998us
W20260709 13:14:41.329005  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.329368  1564 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.332113  5144 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.337126  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:41.249725  5622 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.342118  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.356868  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.298900  5683 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.385147   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:41.343565   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 465562us
I20260709 13:14:41.389421   999 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:41.390868   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.391492   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.388234   848 raft_consensus.cc:1240] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:41.326545  5501 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.349740  5714 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.022s	sys 0.032s
W20260709 13:14:41.396126   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.339886  5814 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.350999  5669 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.393299  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.387844  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:41.388784  5839 log.cc:927] Time spent T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:41.385295  5833 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.401250  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:14:41.394555   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.348170  5625 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:41.404471   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:41.412829   999 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:41.396449  5714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.023s	sys 0.032s
I20260709 13:14:41.133381   750 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.393942  5643 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.411676   818 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.383682  5621 consensus_queue.cc:1048] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:14:41.448314   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.449229  5575 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.391640  5805 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.449936  5575 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.450174   818 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.450950  5575 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.453059  5575 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:41.453857  5849 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:41.454440  5849 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.454807  5849 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:41.456720   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.458792  5501 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.400027  5840 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.460201   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:41.460951  5759 consensus_meta.cc:287] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.718s	user 0.005s	sys 0.004s
I20260709 13:14:41.399807  5703 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.463794  5703 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.464636   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.465397   926 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:41.466406  5703 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:41.474179  5846 log.cc:927] Time spent T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:14:41.476090  5844 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:41.482800   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.483678   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:41.483743   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.438346  5113 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:41.400271  5622 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:41.444185  5683 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.454851  5575 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.458999   924 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:14:41.487455   922 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.010s	sys 0.009s
I20260709 13:14:41.397737  5774 consensus_queue.cc:1059] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:14:41.490962   818 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.512610  5679 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:41.513129  5679 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.513763  5679 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.513019   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.514884  4993 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.515314  4993 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.515697  4993 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.516368  5575 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.506673   922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.011s	sys 0.009s
I20260709 13:14:41.524840   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.525635   924 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
W20260709 13:14:41.530332   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.529767   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.533227  5722 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:41.412690  5778 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.539316  4993 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.543706   967 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.028s	sys 0.041s
W20260709 13:14:41.546996   967 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.028s	sys 0.041s
I20260709 13:14:41.548344  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.548627  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.549209  1003 raft_consensus.cc:2410] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:41.550170  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.551255  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.429539   750 catalog_manager.cc:5697] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 10, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 10 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.553351   750 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 4 to 10, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.555475   750 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.557137   750 catalog_manager.cc:5697] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 9, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:41.543759  5842 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.560374  5774 raft_consensus.cc:3055] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:41.560781  5774 raft_consensus.cc:740] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:41.560526  5842 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.561798   996 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:41.561652  5774 consensus_queue.cc:260] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.562775  5842 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.562875  5774 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.563959   995 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:41.566865  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.459848  5840 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.406198   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:41.519366  5575 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:41.566746   893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.015s	sys 0.019s
I20260709 13:14:41.497062  5696 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:14:41.582516   893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.015s	sys 0.019s
I20260709 13:14:41.583166  4993 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:41.585253   851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.006s	sys 0.021s
W20260709 13:14:41.585830   851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.007s	sys 0.021s
I20260709 13:14:41.412554  5669 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.589300  1598 raft_consensus.cc:1240] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:14:41.558782   750 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:41.590728  5703 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:41.591728  5703 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.591904  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.592108  5703 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.592528  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.593189  1000 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:41.593614  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:41.594105  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.594203  1004 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.594704  1000 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:41.591001   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.402828  5814 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:41.568493  5832 consensus_meta.cc:287] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.851s	user 0.006s	sys 0.005s
I20260709 13:14:41.592525  1002 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
W20260709 13:14:41.590646   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 184539us
I20260709 13:14:41.571678   999 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:41.596043  1001 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:41.598539   896 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.481259  5625 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.605566   895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.010s	sys 0.019s
W20260709 13:14:41.607213   895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.010s	sys 0.019s
W20260709 13:14:41.595511  5790 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.012s	sys 0.017s
W20260709 13:14:41.613737  5708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.033s	sys 0.041s
W20260709 13:14:41.616634  5708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.033s	sys 0.041s
W20260709 13:14:41.617306  1564 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.31.120.2:34547 timed out after 2.053s (SENT) [suppressed 6 similar messages]
I20260709 13:14:41.617739  5810 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.618211  5810 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.620225  5810 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:41.620417  5840 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:41.616122  5790 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.012s	sys 0.017s
I20260709 13:14:41.622828  1004 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:41.623112   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.623888   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.615890   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.624615   847 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 13: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:41.624507   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:41.624740   995 raft_consensus.cc:2376] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13. Re-sending same reply.
I20260709 13:14:41.625836  5840 raft_consensus.cc:2764] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:41.630141   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.636551   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:41.637280   846 raft_consensus.cc:3060] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:41.639192   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.636570  5753 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.643328 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:41.643596  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.647744   846 raft_consensus.cc:2468] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:41.649356  5722 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:41.649999  5722 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.662526   821 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.415686  5775 consensus_queue.cc:237] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.664279  5753 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.666267  5840 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.666569  1599 raft_consensus.cc:1240] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:41.666862  5840 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.669044  5840 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.671788   821 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.672996  5669 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.673918   847 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:14:41.674767  5753 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:41.678048  5814 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.678841  1596 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:41.679590  5840 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:41.680300  5840 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.682750  5840 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:41.687628   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 89150.4us
I20260709 13:14:41.688534  5730 raft_consensus.cc:2804] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:41.689136  5730 raft_consensus.cc:697] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:41.690007  5730 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.549685  5778 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.693423  5778 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.587886  5267 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:41.693714  5714 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.694013  5267 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.504788  5113 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.694500  5714 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.696486  5113 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:41.698740  5625 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.699887  5714 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.701717   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.702318   999 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:14:41.550240  5683 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 13:14:41.704312  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:41.705127  1599 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:41.612995  5832 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.703347  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:41.612448  5847 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.707242  5847 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.708072  5790 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.708617  5790 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.709041  5790 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.709962  5847 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:41.712114  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 94825.4us
W20260709 13:14:41.712684   894 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.713372  5550 ts_tablet_manager.cc:1434] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.643s	user 0.015s	sys 0.004s
W20260709 13:14:41.713687  5550 ts_tablet_manager.cc:1467] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Tablet startup took 643ms
I20260709 13:14:41.714712   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.715317   928 raft_consensus.cc:2468] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:41.715433   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:41.716220   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:41.717381   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.718480   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:41.719661  5302 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:41.720775  5302 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.720970   923 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
I20260709 13:14:41.722049  5832 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.722090   923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42390 (request call id 2377) took 1064 ms. Trace:
I20260709 13:14:41.723011   923 rpcz_store.cc:276] 0709 13:14:40.657991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.675964 (+ 17973us) service_pool.cc:224] Handling call
0709 13:14:41.721948 (+1045984us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004466d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:41.722041 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401602944}
I20260709 13:14:41.723856   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.673528  5669 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:41.724638   928 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:41.726038   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:41.714004  5550 ts_tablet_manager.cc:1469] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Trace:
0709 13:14:33.691406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.700125 (+  8719us) service_pool.cc:224] Handling call
0709 13:14:34.326259 (+626134us) ts_tablet_manager.cc:934] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
0709 13:14:36.102121 (+1775862us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:14:36.993669 (+891548us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:36.993789 (+   120us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:14:37.503024 (+509235us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:14:37.503376 (+   352us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:37.503734 (+   358us) ts_tablet_manager.cc:989] Replica registered.
0709 13:14:37.503797 (+    63us) inbound_call.cc:177] Queueing success response
0709 13:14:39.298040 (+1794243us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:14:41.070560 (+1772520us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:14:41.161720 (+ 91160us) tablet_replica.cc:261] Starting instrumentation
0709 13:14:41.162390 (+   670us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":24213,"mutex_wait_us":75,"raft.queue_time_us":527,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":26305,"tablet-copy.queue_time_us":467511,"tablet-download-pool.queue_time_us":24535,"tablet-download-pool.run_cpu_time_us":1542,"tablet-download-pool.run_wall_time_us":1088079,"thread_start_us":133371,"threads_started":2}
I20260709 13:14:41.727982   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:41.694465  5267 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.733155  1564 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.733574  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-85bbb1b886474a538a2f1a32dd91b560 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:14:41.734233  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-85bbb1b886474a538a2f1a32dd91b560 on tablet 85bbb1b886474a538a2f1a32dd91b560 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:41.734679  1564 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.735975  5280 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:41.736083  5761 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:41.737228  1564 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.739002  5703 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:41.740255   998 raft_consensus.cc:2393] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:41.741819   750 catalog_manager.cc:5697] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.742684   929 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:41.743669   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.752950   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:41.744626   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:41.745932  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.753662   894 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.743319   750 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.755967   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.764210   928 raft_consensus.cc:2410] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:14:41.770304   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 63033us
I20260709 13:14:41.770454   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.771368   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:41.772424  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:41.773257  1001 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:41.772374  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.768813   750 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 11, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:41.773180   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.779076   998 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:41.781391   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.783354   750 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.780560   896 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:41.785328   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.786299   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:41.787817  1599 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:41.784840  5778 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.789811   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.765004  5847 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:41.818948   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.769482  5761 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:14:41.776695  5814 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.824213   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.809692   821 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:41.738544  5267 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.830169  5861 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.819563   894 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:41.820727   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.827107   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.788525  5703 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:41.835933   853 raft_consensus.cc:2410] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:41.812619  5625 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:14:41.811537   895 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.838332   851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46364 (request call id 2217) took 1178 ms. Trace:
I20260709 13:14:41.838901   851 rpcz_store.cc:276] 0709 13:14:40.659719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.659893 (+   174us) service_pool.cc:224] Handling call
0709 13:14:40.660245 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:40.660604 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.660650 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.661139 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:14:41.586517 (+925378us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:41.586576 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.586670 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.838105 (+251435us) raft_consensus.cc:1685] finished
0709 13:14:41.838134 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.838271 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1115,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":14441,"thread_start_us":924995,"threads_started":1,"wal-append.queue_time_us":1175390}
I20260709 13:14:41.830883   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.834203   930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 2370) took 1186 ms. Trace:
W20260709 13:14:41.865458   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 54082.6us
I20260709 13:14:41.866554  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.817595  5753 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.818338  5778 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.871711  1004 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 13: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:41.866554   895 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.867928   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.870126   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:41.870182   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 50857us
I20260709 13:14:41.877493   928 raft_consensus.cc:1275] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:41.865087   821 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:41.873725   851 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:41.882347   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.887089   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:41.887748   998 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:41.832154  5862 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:41.892194   921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.050s	sys 0.059s
I20260709 13:14:41.871479  5625 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:41.864495  5322 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:41.895524   921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.050s	sys 0.059s
I20260709 13:14:41.898041   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:41.880656  5557 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:41.895390   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 360674us
I20260709 13:14:41.880502  5778 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.867712  5703 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:41.835245  5849 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.958109   921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46362 (request call id 759) took 1395 ms. Trace:
W20260709 13:14:41.960559   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51306.7us
I20260709 13:14:41.817785  5790 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:41.962563   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:41.963397   851 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.889084  5832 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.830909  5473 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:41.966756  5703 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:41.970057  5863 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:14:41.897092  5625 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.892589  5113 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.883049  5302 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:14:41.952929  5557 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:14:41.966053   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.980917   969 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.898159  5862 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:41.997799  5862 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:41.998756  5849 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.999563  5842 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:42.002321  5866 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:41.864869  5861 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.003368  5855 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:41.963738  5556 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.884436  5753 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.012676   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.013002   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.013257   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.013669   923 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:42.014277   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:41.996470   927 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
W20260709 13:14:42.004165   926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.012s	sys 0.042s
I20260709 13:14:41.996960   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:41.896131  5679 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:41.989820  5625 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.014384   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:42.015761   926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.013s	sys 0.042s
I20260709 13:14:41.974985   851 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:42.005427   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:42.032276   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:41.986711  5267 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.007544  5810 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:42.045940   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.005336  5557 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:42.040328  5859 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:42.105298  5862 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.106902   928 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:42.108978  5862 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:41.995266  5113 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.067050  5805 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:42.074815  5840 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:42.076529  5267 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:41.841689   930 rpcz_store.cc:276] 0709 13:14:40.647398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.739410 (+ 92012us) service_pool.cc:224] Handling call
0709 13:14:40.739707 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.739916 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:41.314989 (+575073us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.833882 (+518893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:41.833936 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:41.833983 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.834045 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.834149 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126359,"prepare.queue_time_us":260702,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":5039,"raft.queue_time_us":26070,"thread_start_us":918430,"threads_started":2}
I20260709 13:14:42.115224  5823 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.112370  5621 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:42.117872  5805 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.958639   921 rpcz_store.cc:276] 0709 13:14:40.562728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.562913 (+   185us) service_pool.cc:224] Handling call
0709 13:14:40.563261 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.563478 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:40.618350 (+ 54872us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.937005 (+318655us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b1800382b40. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d0891ade1 00007f0d089275b1 00007f0cffa34e07 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d026d920b 00007f0d026ded23 00007f0d026df23a 00007f0d02b49a2a
0709 13:14:41.957662 (+1020657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:41.957732 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:41.957795 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.957883 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.958040 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56430,"prepare.queue_time_us":54709,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":56593,"spinlock_wait_cycles":41074432,"thread_start_us":963916,"threads_started":1}
I20260709 13:14:42.195526   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:42.131239  5708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.010s	sys 0.015s
I20260709 13:14:42.121605  5761 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:42.270033   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 77360.6us
I20260709 13:14:42.275102  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.276722  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.278584  1564 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.130947  5714 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:42.182209  5862 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.181353  5840 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:42.191910  5849 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.299551  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:42.204320  5805 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:42.308912  1004 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:42.286273  5868 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.264535  5730 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.186275  5267 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:42.322410   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 278546us
I20260709 13:14:42.329651   996 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:42.330425   996 raft_consensus.cc:606] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:42.330857   996 raft_consensus.cc:614] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:42.331815  1004 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:14:42.333841  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:42.293895  5708 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.205750  5823 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.381089  5753 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.382308  5879 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.384003  5840 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.393743  5280 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:42.394207  5280 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:42.394753  5113 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.374732  5708 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.401041  5845 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:42.402431  5708 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.405289  5267 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.326620  5790 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.327239  5718 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.407245  5718 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.409240  5718 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.319137  5849 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.410225  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:42.410797  1004 raft_consensus.cc:606] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.411152  1004 raft_consensus.cc:614] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:42.411337  5849 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.411742  5473 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.326303  5868 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.412895  5280 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.413914  5868 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.414158  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:42.309489  5862 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.414734  1004 raft_consensus.cc:606] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:42.415076  1004 raft_consensus.cc:614] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:42.414064  5280 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:42.415974  5862 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:42.417991  5622 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.881s	user 0.007s	sys 0.002s
W20260709 13:14:42.418442  5865 log.cc:927] Time spent T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
W20260709 13:14:42.420200  5856 log.cc:927] Time spent T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:14:42.290716  5761 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:42.432768   821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.027s
W20260709 13:14:42.433233   821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.004s	sys 0.027s
W20260709 13:14:42.433624   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 552755us
I20260709 13:14:42.435738  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:42.436511  1004 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:42.382956  5879 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.343248  5703 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:42.443061   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 410847us
I20260709 13:14:42.370476  5861 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:42.449683  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 129872us
I20260709 13:14:42.476485   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.502051   930 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
W20260709 13:14:42.454294  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 175778us
I20260709 13:14:42.481107   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:42.502969   929 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:42.477377   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:42.475617   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.480327   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:42.503911   928 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:42.487095   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.487298   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.451121   969 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:42.342114  5730 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.488029  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:42.565320  5886 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.445770  5866 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:42.500259   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.459111   924 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:14:42.452538   967 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.502249  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:42.601791   997 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:42.485369   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.535816   996 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:14:42.531869  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:42.603111  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 71356us
I20260709 13:14:42.508919   927 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 16: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:42.473064  5761 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.509164   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.555140   929 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:42.469115  5679 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:42.508798  5722 consensus_meta.cc:287] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.858s	user 0.014s	sys 0.000s
I20260709 13:14:42.572515   928 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:42.457477  5625 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:42.573969  5874 log.cc:927] Time spent T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.232s	user 0.005s	sys 0.000s
W20260709 13:14:42.576930  5880 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:14:42.622390  5867 log.cc:927] Time spent T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.296s	user 0.003s	sys 0.000s
W20260709 13:14:42.577764  5873 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:14:42.623766  5722 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.625232   926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42390 (request call id 2387) took 1146 ms. Trace:
W20260709 13:14:42.625545  5871 log.cc:927] Time spent T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:14:42.578516  5888 log.cc:927] Time spent T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:14:42.626166  5805 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.582204  5823 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.562819  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.598614  1004 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
W20260709 13:14:42.490458  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.566027  5714 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.629586  5714 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.587664  5565 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:42.632401  5879 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:42.605325  5881 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:42.635591  5892 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:42.636261  5892 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:42.637321  5892 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:14:42.613204   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 162206us
W20260709 13:14:42.616240   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 163788us
I20260709 13:14:42.617353  1598 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:42.620393  5730 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.619910  5703 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:42.605131  5886 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.622375  5625 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.612922  5621 consensus_queue.cc:1048] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:14:42.643550   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.646903  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.648046  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.652978  1003 raft_consensus.cc:2368] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:42.626219  5722 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.656770  5866 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:14:42.666956  5557 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: Time spent flushing consensus metadata: real 0.591s	user 0.007s	sys 0.004s
I20260709 13:14:42.642489  5565 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.673259  5557 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.675622  5886 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:42.677955  5895 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
I20260709 13:14:42.680505  5714 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:42.682361  5778 consensus_meta.cc:287] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.003s
I20260709 13:14:42.682876  5778 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.688093  5778 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.689936  5805 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:42.675979  5557 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:42.566519   750 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.007s	sys 0.015s
W20260709 13:14:42.725761   750 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.007s	sys 0.016s
I20260709 13:14:42.731449   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.461776   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:42.737574   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 275724us
I20260709 13:14:42.740864   846 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:42.744395   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.748693   818 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:42.594507   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 61626.2us
I20260709 13:14:42.752132   896 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:42.754906   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:42.759677   896 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:42.760135   896 leader_election.cc:403] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:14:42.764955   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 110565us
I20260709 13:14:42.625706   926 rpcz_store.cc:276] 0709 13:14:41.478805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.478975 (+   170us) service_pool.cc:224] Handling call
0709 13:14:41.490707 (+ 11732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:41.491173 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:41.491215 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:41.491819 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:14:42.043020 (+551201us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:42.043058 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:42.043119 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:42.621877 (+578758us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0d026da86e 00007f0d026d9bb6 00007f0d026d97d8 00007f0d026e3221 00007f0d026e31d9 00007f0d026e3169
0709 13:14:42.624905 (+  3028us) raft_consensus.cc:1685] finished
0709 13:14:42.624928 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:42.625151 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15797,"raft.queue_time_us":1176,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":51074,"spinlock_wait_cycles":77728512,"thread_start_us":550874,"threads_started":1,"wal-append.queue_time_us":797985}
I20260709 13:14:42.765552   928 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:42.766322   923 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
W20260709 13:14:42.768769   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 125203us
I20260709 13:14:42.770346   967 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.772840   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.774020   999 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
W20260709 13:14:42.776598  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.773000  1002 raft_consensus.cc:606] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.784602  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:42.785943  1599 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 13: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:42.775831   819 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:42.792975 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:42.796190  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:42.799769   967 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.799958  5113 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:42.810343  5501 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:42.811204  5113 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.811578   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.812742   853 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:42.813750  5501 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:42.814702  5501 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:42.815774  5501 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.814301  5113 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.820530   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:42.820930  5113 raft_consensus.cc:2764] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:14:42.804905  5302 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:42.823002  5557 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:42.799284   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:42.824092  5302 raft_consensus.cc:697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:42.825209   995 raft_consensus.cc:2393] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 13: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:42.825907   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:42.825889  5302 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:42.805384   924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46362 (request call id 768) took 1091 ms. Trace:
I20260709 13:14:42.806363   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.827570   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:42.828459   847 raft_consensus.cc:2393] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 11: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:42.828544  5886 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.827277   924 rpcz_store.cc:276] 0709 13:14:41.713935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.715197 (+  1262us) service_pool.cc:224] Handling call
0709 13:14:42.804896 (+1089699us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400090454. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:42.805132 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616620800}
I20260709 13:14:42.829572  5886 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:42.833564  5886 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.807186  5267 raft_consensus.cc:2804] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:42.807957  5473 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:42.809150  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:42.809592  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.823021  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:42.847752  1598 raft_consensus.cc:2368] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:42.819442  5501 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:42.843910  5267 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.844709  5473 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:42.845623  5113 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:42.873528  5912 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.174s	user 0.000s	sys 0.002s
I20260709 13:14:42.875928  5759 raft_consensus.cc:993] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:42.876590  5759 raft_consensus.cc:1081] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:42.878793  5859 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:42.879223   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79744.4us
I20260709 13:14:42.880406   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:42.882355   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:42.883296  5859 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:42.883968  5577 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:42.884212  5859 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:42.858369  5267 raft_consensus.cc:3060] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:42.863957  5473 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:42.901583 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:42.890823  5859 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.904450   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.905067   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:42.905992   851 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:42.906790   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.907189   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.908099   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.908396   846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46360 (request call id 2693) took 1042 ms. Trace:
I20260709 13:14:42.675822  5565 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:42.921744   995 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:42.923439  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:42.929395  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:14:42.937036  5914 log.cc:927] Time spent T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.081s	user 0.000s	sys 0.002s
I20260709 13:14:42.942011   846 rpcz_store.cc:276] 0709 13:14:41.865740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.866692 (+   952us) service_pool.cc:224] Handling call
0709 13:14:42.908101 (+1041409us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540019f0d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:14:42.908329 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994713856}
I20260709 13:14:42.957713   848 raft_consensus.cc:2368] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:42.961077   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:42.948735  5859 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.964439   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:42.973703   851 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:42.967005   854 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:42.978350   846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:42.979257   846 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:42.983745  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:42.984511  1000 raft_consensus.cc:2368] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:42.995211   846 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:42.999971  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:43.001533  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 72396us
I20260709 13:14:43.003953   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:43.005798   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 120374us
I20260709 13:14:43.013253   851 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:42.993952  5714 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.019944   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:43.020041  5714 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.022522  5714 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.024652   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:43.028333  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 93922.3us
I20260709 13:14:43.040711  5621 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:43.041266  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.052335   846 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.054989   846 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:43.065169   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52649.5us
I20260709 13:14:43.067160   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:43.068434  5915 log.cc:927] Time spent T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:43.074900   896 leader_election.cc:400] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:43.097824   896 leader_election.cc:403] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:14:43.076579  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:43.081025   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.081552   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.084028   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.084221   895 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.084741   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.086117   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:43.087287  5892 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.090255  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:43.056190   850 raft_consensus.cc:1275] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:43.071187  5267 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.101279  5892 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.103904   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.104308  5889 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:43.104839  5889 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:43.105316  5889 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:43.108078  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 101999us
I20260709 13:14:43.108661  5901 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:43.109220  5774 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.110594  5901 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:43.114940  5899 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:43.115563  5891 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.112528  5302 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:43.115562  5778 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.117741  5891 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.118468  5778 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.119282  5892 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.116883  5899 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.095500  5621 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.115892   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.124720  5778 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.124845  5862 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.127851  5899 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.130439  5899 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:43.133484   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66460us
I20260709 13:14:43.137436  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.140985  5267 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.142712   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.143985   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.144200  5901 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.144275  5810 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.145533   853 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 16: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:43.145113  5810 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.145308   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.146091   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.146730   895 leader_election.cc:400] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
W20260709 13:14:43.148343   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 123891us
I20260709 13:14:43.149185   818 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:43.151738   894 leader_election.cc:400] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:43.152091   894 leader_election.cc:403] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:14:43.150631  5267 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:43.152631   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 52161.5us
I20260709 13:14:43.154175  5714 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.155529  5577 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.156164  5577 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.156905  5840 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.157523  5840 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.158391  5577 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.162359  5862 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12.
I20260709 13:14:43.163468   853 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:43.163358   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.163990  5840 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.164906  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:43.165714  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:43.122295  5621 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.167431   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.167913  5774 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.136729  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.134351  5501 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:43.151185   818 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:43.149223   895 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.187845  1593 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:43.188097   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.189548  5840 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.183115  1000 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:43.197396   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:14:43.198056   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 123516us
I20260709 13:14:43.199049  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
W20260709 13:14:43.200381   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 49346.4us
I20260709 13:14:43.174273  5810 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:43.202709  5113 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:43.204007  5501 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:43.199635  5840 consensus_queue.cc:1059] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:14:43.196098  5473 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.217756   896 leader_election.cc:400] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:43.219995  5557 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 13. Current term is 14.
I20260709 13:14:43.212797  5556 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:43.233459   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 84318.4us
I20260709 13:14:43.211244  5805 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:43.236701  5805 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.237126  5805 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:43.237375  5280 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:43.195801  5774 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.241150  5859 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.235879  5840 raft_consensus.cc:3055] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:43.242453  5840 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:43.242952   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.243626   849 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:14:43.243490  5840 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.245178  1566 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:43.246486  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.247921  5899 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.248512  5899 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.249496  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.250104   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.250133  5280 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:43.223120  5501 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.251940  5840 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.251578  5280 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.210388  5891 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.241753  5621 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:43.255669   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50174.2us
I20260709 13:14:43.256434  1600 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:43.259210  5881 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:43.259642  5881 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.211206  5730 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:43.260121  5881 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.260694  5621 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.261224   853 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:43.261336   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 45786.6us
I20260709 13:14:43.264067  5810 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.264649  5473 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.264734  5810 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.259675  5901 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:43.241533  5774 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.215713  5113 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:43.267190  5810 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.271350   896 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:43.284222  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.284816  1003 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.289551  1598 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Advancing to term 14
I20260709 13:14:43.290659   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.291417   926 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.291884   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:43.282652  5879 consensus_queue.cc:1059] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:14:43.293287  5891 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.299553   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.280941   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.267058  5901 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.298331  5730 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:43.312526   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.301443  5810 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:43.313145  5730 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.314847  1000 raft_consensus.cc:2410] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:14:43.314934  5730 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:43.315814  5899 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.316537  5557 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:43.317190  5557 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.319298  5557 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.319995  5810 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:43.341202   924 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:43.340716  5810 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.342187  5810 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:43.307842  5891 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.327636  5859 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.328058   921 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:43.330950  5113 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.334602  5714 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689) with higher term. Message: Invalid argument: T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:43.334672  5886 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.338477   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.339079   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.323889   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.369308  1003 raft_consensus.cc:2468] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
W20260709 13:14:43.371853   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 63308.8us
I20260709 13:14:43.376655   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.377314   921 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:43.380606  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 258826us
I20260709 13:14:43.388018   923 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 11.
I20260709 13:14:43.390821  5891 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:43.393043   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.396091   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:43.397298  5886 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:43.390405  5805 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.402052  1567 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.407294  5845 raft_consensus.cc:2804] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:43.407833  5845 raft_consensus.cc:697] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:43.397818  5886 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:43.408776   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.393872  5881 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.432600  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.448779   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.429769  5886 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.455822   895 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.452692   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:43.459492  1000 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.469580  5885 consensus_queue.cc:1059] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:43.472311  5885 raft_consensus.cc:3055] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:43.472692  5885 raft_consensus.cc:740] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:43.473621  5885 consensus_queue.cc:260] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.475076  5885 raft_consensus.cc:3060] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.481259   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:43.482086   853 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:43.483980   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.489748   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.490738   922 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 9.
W20260709 13:14:43.497293   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 44694.7us
I20260709 13:14:43.500918  5557 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:43.505919  5714 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:43.513967   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:43.517390  1598 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:43.459219  5901 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:43.524572   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 88767.7us
I20260709 13:14:43.408805  5845 consensus_queue.cc:237] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.528765  5805 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:43.530329   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.531155   928 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:43.533342  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:43.537252   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.537477  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.538386   998 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 10: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:43.538389  1003 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:43.548480   922 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:14:43.557848   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.510106  5889 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.568113  5840 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:14:43.577634   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 52174.4us
I20260709 13:14:43.588649   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.589696   851 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:43.568415  5557 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:43.592427   998 raft_consensus.cc:1240] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:14:43.594676   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.591735  5840 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.595935  5862 consensus_queue.cc:1048] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:43.568408  5859 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:43.597328  5840 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.599361   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:43.600968   849 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.602543   928 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 17.
I20260709 13:14:43.601954  5473 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:43.619287   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.624840   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.614508  5881 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:43.627302   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.627213   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.622013   926 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:43.628952   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:43.643064  1003 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:43.645510   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:43.640100  5302 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:43.646675   927 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:14:43.646543  5928 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:14:43.649971   921 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 10.
I20260709 13:14:43.629825  5889 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.675905   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.676288  1001 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:43.689744   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.690928   998 raft_consensus.cc:2393] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 16: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:43.693888  5929 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:14:43.698196  5930 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.146s	user 0.002s	sys 0.000s
I20260709 13:14:43.701583  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.702433  1001 raft_consensus.cc:2393] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64f2cb5b0e174002811ef197f9eec829 in current term 13: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:43.705684  1000 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:43.726351   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:43.729468   853 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:43.733376  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.735176  5840 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:43.735847  5840 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.736404  5840 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.737702  5886 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.751514   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.751914  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.755497   821 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:43.755774   818 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:43.756078  5907 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:43.756314  5113 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:43.776492  5113 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:43.763897   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:43.776959  5113 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:43.777184   927 raft_consensus.cc:3060] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.768296   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:43.770318  5267 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.774518  5855 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:43.781213  5855 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:43.782734  5730 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.784444   929 raft_consensus.cc:1240] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:43.786684  5730 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:43.792886   818 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:43.794384  5885 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:43.794869  5885 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.795420  5885 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.796393  5907 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:43.794836   995 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 17.
I20260709 13:14:43.814661   921 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:43.816538  1000 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:43.823040   995 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:43.827385   927 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:43.834357   849 raft_consensus.cc:2468] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:14:43.828946  1564 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:43.843101   818 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:43.806974  5473 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.815985  5810 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:14:43.845217  5557 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.842706  5774 raft_consensus.cc:2804] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:43.877264   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 231835us
W20260709 13:14:43.886979  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 228305us
I20260709 13:14:43.908028  5925 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:43.882334  5774 raft_consensus.cc:697] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:43.909533  5925 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:43.910095  5925 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:43.910398  5774 consensus_queue.cc:237] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.906319  5113 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.879101  5557 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.868023  5855 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.968204   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.911194  5925 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:43.993871   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:43.996058  5866 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:43.996668  5866 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:43.997134  5866 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:43.997875  5840 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:43.996070   995 raft_consensus.cc:2393] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:14:43.999974   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 163777us
I20260709 13:14:43.957916  5556 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:14:44.007001  5473 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:44.012676  5840 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:43.982810  5886 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.005137   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:44.034662   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 196969us
I20260709 13:14:44.036006   818 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:44.040001   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 162550us
W20260709 13:14:44.041261   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:44.046504   820 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.015118  5473 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.033221  5907 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:44.080302  5113 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:44.084283  5944 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:44.094007  5907 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:44.096845  5473 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.104888  5669 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:14:44.119715  5845 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.120420  5845 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:44.122563  5845 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.132697  5805 raft_consensus.cc:1064] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
W20260709 13:14:44.133235  5805 raft_consensus.cc:1068] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Unable to promote non-voter 597687e464174d1ea6043161e037d240: Service unavailable: leader transfer in progress
I20260709 13:14:44.133713  5557 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:44.139881  5885 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.142699  5885 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.145054   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:44.157668   854 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:44.149353  5855 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:44.157891   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 121960us
I20260709 13:14:44.149914   853 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:44.150991 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:44.160923   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 127134us
I20260709 13:14:44.168983   821 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:44.162636   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:44.152726  5925 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:44.154680  5501 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:44.172112  5501 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.174577  5501 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:44.180610  5885 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:44.181318  5885 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:44.182299  5885 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.183960  5944 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:44.187064   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.187076  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:44.188016   926 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:44.188161  1599 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 13.
I20260709 13:14:44.190384   818 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:44.209709  1003 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:14:44.210297  1003 pending_rounds.cc:85] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 7
I20260709 13:14:44.216584  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:44.219671   854 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:44.221908   850 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:44.203004   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.223532   924 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 16: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:44.222234   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:44.242620   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 46640.9us
I20260709 13:14:44.248498   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:44.210600  1003 pending_rounds.cc:107] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:14:44.249403   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:44.252483   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:44.254616   926 raft_consensus.cc:1275] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:14:44.255273   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:44.251549   818 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:44.248785   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:44.249621  1003 raft_consensus.cc:2889] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.215857  5953 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:44.261113   850 raft_consensus.cc:2368] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:44.255020   926 pending_rounds.cc:85] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 11
I20260709 13:14:44.267735   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:44.270167   926 pending_rounds.cc:107] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:14:44.270637   926 raft_consensus.cc:2889] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.271389   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:44.272071   999 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
W20260709 13:14:44.305140   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 52728.3us
I20260709 13:14:44.243152  5774 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:44.329423   855 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:14:44.336773  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:44.341109   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 89637.5us
I20260709 13:14:44.342727   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.319464  5473 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:44.277806  5953 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:44.258143  5840 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:14:44.327347  5774 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.332358  5907 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:44.265638  5866 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.351346  5473 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:44.362083   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 108874us
I20260709 13:14:44.368893   969 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:44.373660   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:44.368921   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.391040  5953 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.394389  5952 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:44.394966  5952 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.396085  5866 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.397043  5952 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.397790   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:44.398378   997 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:44.399708  5945 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.127s
W20260709 13:14:44.401652   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.371767  5473 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:44.361972  5774 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:44.408921   849 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:44.460394   995 raft_consensus.cc:1275] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:44.413173   818 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:44.383082  5810 raft_consensus.cc:2804] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:44.474845  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:14:44.476339   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 63281us
I20260709 13:14:44.481060   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.417769  5907 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:44.499063   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:44.500988  5957 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 13:14:44.507953   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.508714   922 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 15: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:44.509495   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.510507   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.510988   922 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:44.476686  5945 raft_consensus.cc:2764] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:14:44.511724   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.512784   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.513283   921 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:44.513310  5907 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.514922   929 raft_consensus.cc:1240] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:44.515904   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:14:44.516186  5960 log.cc:927] Time spent T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:14:44.517024   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.517997   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.532339   924 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:44.469594  5669 consensus_queue.cc:1059] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:14:44.539561   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.540138   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.544168   924 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:44.493836  5810 raft_consensus.cc:697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:44.577491   927 raft_consensus.cc:1240] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:44.489043  5946 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:14:44.573601   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.576231   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.580482   929 raft_consensus.cc:3060] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:44.559739   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.492070  5935 consensus_queue.cc:1048] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:14:44.584033   928 raft_consensus.cc:2393] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 10: Already voted for candidate 64f2cb5b0e174002811ef197f9eec829 in this term.
I20260709 13:14:44.587556   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:14:44.651811   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53219us
I20260709 13:14:44.654803   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.655943   854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:44.657769  5810 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.659142   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:44.660624  5961 log.cc:927] Time spent T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:14:44.661095   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:44.541066  5962 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:14:44.666103   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:44.690387   928 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:44.691639   921 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:14:44.669579   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:44.707206   997 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:44.681416  5953 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:14:44.670821   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:14:44.672339  5925 consensus_meta.cc:287] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
I20260709 13:14:44.672984  5805 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:44.673076   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 85599.4us
I20260709 13:14:44.675311  5840 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.675863  5936 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:44.679674   926 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:14:44.682093   895 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:44.746027   927 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:44.661100  5473 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:44.687964   929 raft_consensus.cc:2468] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:44.737308   923 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:44.738976   928 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:44.741253   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:44.787595   926 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:14:44.787470   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 105477us
I20260709 13:14:44.789844   895 leader_election.cc:400] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:44.790294   895 leader_election.cc:403] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:14:44.786024  5845 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.806535   895 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:44.808763   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:44.810509   895 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:44.790939  5267 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:44.799636  5556 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:44.737676  5953 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.767230  5840 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.817829  5886 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:44.823805  5952 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:44.859007   921 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:44.774181  5936 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:44.819252  5965 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 12. Current term is 13.
I20260709 13:14:44.846199  5983 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:14:44.859300  5730 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:44.804373  5845 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:44.917013   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 94162.2us
I20260709 13:14:44.860832  5955 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.227s
W20260709 13:14:44.922663   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 76242.7us
W20260709 13:14:44.943783   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.955807  5862 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.956382  5862 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.958421  5980 raft_consensus.cc:1064] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
W20260709 13:14:44.958904  5980 raft_consensus.cc:1068] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Unable to promote non-voter 597687e464174d1ea6043161e037d240: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:14:44.960427  5625 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:44.960858  5625 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:44.961189  5953 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.961654  5625 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:44.963136  5625 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:44.958843  5473 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:14:44.930537   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 89075.7us
I20260709 13:14:44.960386  5840 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:44.926903  5730 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:44.974360   846 raft_consensus.cc:1275] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:14:44.991178  5979 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:14:44.975168  5970 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:14:45.006628   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 153860us
I20260709 13:14:44.958425  5862 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:44.934154  5774 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.017009  5935 consensus_queue.cc:1048] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:45.022919  5845 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.023268  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.023821  5936 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:45.023975  1000 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:45.025446  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:45.026825  5935 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.027238  5935 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.027586  5935 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.034523   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:45.035298   851 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:45.042827   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.043382  5714 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.043748   929 raft_consensus.cc:2393] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 10: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:45.044023  5714 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:45.047587  5714 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.048421  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:45.049480  5891 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:45.049881  5891 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:45.050483  5774 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.051031   921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.050730  5891 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:45.051815  1600 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:14:45.052996  5805 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:45.051663   921 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:14:45.063817   849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:45.065478  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
W20260709 13:14:45.076491   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56166.8us
I20260709 13:14:45.077310   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.083357  5267 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:45.084806  5267 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:45.100746   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.101595   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:45.101979  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:45.103672   929 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 14: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:14:45.103864  1004 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.110579   924 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:14:45.117165   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 334787us
W20260709 13:14:45.117918   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.118335   967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.121901   818 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.150271   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.154850  5625 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:45.155035  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:45.041397  5730 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:45.159829   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.082634  5621 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.102999  5936 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:45.119694  5557 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.173429   995 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:45.162644  5891 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.152041  5885 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:45.171833  5957 log.cc:927] Time spent T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:14:45.188596   999 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:14:45.192425   968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.167963  5936 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:45.168146  5714 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:45.204741   967 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.172549  5621 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:45.224193  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57289.4us
I20260709 13:14:45.209549  5936 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.225977   967 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.229332   967 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.216387  5840 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.201339  5962 consensus_queue.cc:1048] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:14:45.242592  5621 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.235011   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.238994   967 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.248550   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.249132  5985 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:45.240144   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.249868  5985 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:45.252465  5556 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.237707  5805 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:45.216964  5714 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:45.261039   853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:45.260011   967 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.270362   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.270967   929 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
W20260709 13:14:45.271957   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 118650us
I20260709 13:14:45.274330  5945 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:45.276861   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:45.277173  5947 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.278477  5999 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:45.278985  5999 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:45.279254   929 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:14:45.280426   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 116847us
I20260709 13:14:45.281162  5999 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.286211   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:14:45.287477   896 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.287845  5901 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:45.306584   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:45.389108  6002 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.084s	user 0.001s	sys 0.003s
I20260709 13:14:45.392684  5714 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:45.397009   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 92972.8us
I20260709 13:14:45.399106  5935 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.399690  5994 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.400355  5994 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.405681   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
W20260709 13:14:45.406915   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 98494.9us
I20260709 13:14:45.407035  5985 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.406829   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:45.408319   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 148386us
I20260709 13:14:45.408955   999 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:14:45.411725   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:45.410249  5994 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:45.408893   896 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.422885   848 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:45.443279  1567 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:45.424284  5714 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.431653  5855 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:45.409044   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:45.414541  1004 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:14:45.436389   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.440550  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:45.452651  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.442386   925 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:45.458662   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:45.459489  5999 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:45.459767  5845 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:45.465101  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:45.474298   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.475811  5845 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:45.476810  5845 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.480140  5947 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:45.481242  5991 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:45.480790  5947 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:45.481698  5964 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:45.482640   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.482445  5947 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.452648  6010 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:45.459964  5999 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:45.493332   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 84368.4us
I20260709 13:14:45.496656   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:45.483363  5991 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:45.499516   967 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.500818  5985 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.502115  5840 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.502408   967 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.503481  5985 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:45.504503   967 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:14:45.505805  5985 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:45.502570  5840 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:45.505599   967 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.492057  5557 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.448038  5936 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.493512  6010 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.512060   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.498304  5999 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:45.513202  5980 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.507844  5985 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:45.552017  1599 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:14:45.588667   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:45.510912  5840 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.589538   922 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.552706  5501 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:45.550315  5956 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:45.555140  5971 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:45.521075  6010 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:45.576745   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.578040  5935 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:45.579092  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:45.581099  5936 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:45.583003   848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:45.600538   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:45.603142   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:45.611970   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:45.603968   923 raft_consensus.cc:2376] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12. Re-sending same reply.
I20260709 13:14:45.614706   927 raft_consensus.cc:2393] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 15: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:14:45.695299   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 88750.7us
I20260709 13:14:45.700191   821 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.700533  5985 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.702410  5946 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:45.704618  5855 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:45.701025  5885 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.732815   922 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:14:45.734688   967 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.735258  1564 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:45.729178   848 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:14:45.739392  1566 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:45.746249   997 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:45.753000  1004 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:45.756973   967 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:45.747854  5926 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:45.760562  5881 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:45.779881  5947 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.776415  5926 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:45.836498  5926 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.789762  5881 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:45.840505  5881 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.863102  5947 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:45.832075  6017 log.cc:927] Time spent T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:14:45.832969  6020 log.cc:927] Time spent T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:14:45.833887  6019 log.cc:927] Time spent T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:14:45.835074  6024 log.cc:927] Time spent T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:14:45.888383   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 131515us
I20260709 13:14:45.843298  6010 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:45.899660   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 161106us
I20260709 13:14:45.881300  5999 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:45.899757  5956 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.944473   855 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:45.904269  5945 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:14:45.903213  5885 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:45.959298   930 raft_consensus.cc:2368] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11.
I20260709 13:14:45.975581   929 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:14:45.980032  6016 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
W20260709 13:14:45.996539  6021 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.458s	user 0.004s	sys 0.000s
I20260709 13:14:45.998896  5999 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:46.000435  5962 raft_consensus.cc:1064] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:14:46.002823  5840 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.012429   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:46.013208   852 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:46.015336  6027 log.cc:927] Time spent T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:14:46.015578   995 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Rejecting Update request from peer 64f2cb5b0e174002811ef197f9eec829 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:46.018586  6010 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:46.020331  5840 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:46.022136   751 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 9 to 12, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:46.046955   969 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:46.058514   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.058187   751 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 11 to 14, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:46.062780   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
W20260709 13:14:46.062920   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 51037.8us
I20260709 13:14:46.065551  6030 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:46.073721  1001 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:46.076277  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:46.084789   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:46.085485   924 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:46.097496   751 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 7 to 8, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 8 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:46.101209  5774 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:46.106323  1564 leader_election.cc:400] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11.
I20260709 13:14:46.112480   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:46.119020   852 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:14:46.107102  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:46.193812  1600 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:46.121837   750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55328 (request call id 225) took 7272 ms. Trace:
I20260709 13:14:46.106397   995 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:14:46.123052   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 71509.7us
I20260709 13:14:46.205296   927 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:46.125757   751 catalog_manager.cc:5697] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 6 to 7, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.138531   924 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 17.
I20260709 13:14:46.147542   928 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:46.150636  5937 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:14:46.158152  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 44704.7us
I20260709 13:14:46.174777  5955 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:46.189792  5945 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:46.250562  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:46.243157   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:46.230636   852 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:46.231174   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:46.272104  5730 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:46.272773  5730 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:46.273715  5730 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.277153  5947 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.138816  5962 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.297153   928 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:14:46.313704   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53744.9us
I20260709 13:14:46.279671  5774 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:46.314833   751 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:46.258384  5891 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:14:46.351325  5988 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:46.355070   847 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:46.357312  6021 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:14:46.318320  6010 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:46.360606  5774 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:46.361539  5774 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.220929   750 rpcz_store.cc:276] 0709 13:14:38.849438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.849716 (+   278us) service_pool.cc:224] Handling call
0709 13:14:46.121738 (+7272022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:42.627376 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:42.628023 (+   647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:42.628062 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:14:42.628079 (+    17us) tablet.cc:662] Decoding operations
0709 13:14:42.630243 (+  2164us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:42.630311 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:42.630346 (+    35us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:14:42.692618 (+ 62272us) tablet.cc:701] Row locks acquired
0709 13:14:42.692648 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:14:42.692849 (+   201us) write_op.cc:270] Start()
0709 13:14:42.693123 (+   274us) write_op.cc:276] Timestamp: P: 1783602882692781 usec, L: 0
0709 13:14:42.693166 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:14:42.694260 (+  1094us) log.cc:844] Serialized 11318 byte log entry
0709 13:14:43.190651 (+496391us) op_driver.cc:464] REPLICATION: finished
0709 13:14:43.403035 (+212384us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b4400022b4c. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0cff8dadb1 00007f0cff978c85 00007f0cff9796a2 00007f0d026eb1db 00007f0d026e6aaf 00007f0cffa350ac 00007f0d0888fab9 00007f0cffa5169b 00007f0cffa4f93f 00007f0cffa51c02 00007f0d06575ef0
0709 13:14:43.612573 (+209538us) write_op.cc:301] APPLY: starting
0709 13:14:43.612694 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:43.998412 (+385718us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:43.998457 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:44.943253 (+944796us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:44.991929 (+ 48676us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 13:14:44.991993 (+    64us) write_op.cc:312] APPLY: finished
0709 13:14:44.992669 (+   676us) log.cc:844] Serialized 252 byte log entry
0709 13:14:45.212133 (+219464us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:45.212634 (+   501us) write_op.cc:454] Released schema lock
0709 13:14:45.212789 (+   155us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:45.283072 (+ 70283us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:45.283108 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":701418,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":417463,"apply.run_cpu_time_us":136770,"apply.run_wall_time_us":1617155,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125347,"num_ops":29,"prepare.queue_time_us":656280,"prepare.run_cpu_time_us":10773,"prepare.run_wall_time_us":71517,"raft.queue_time_us":180237,"raft.run_cpu_time_us":150632,"raft.run_wall_time_us":412542,"replication_time_us":497230,"spinlock_wait_cycles":14858624,"thread_start_us":1482661,"threads_started":5,"wal-append.queue_time_us":312755,"wal-append.run_cpu_time_us":33600,"wal-append.run_wall_time_us":1424998}]]}
I20260709 13:14:46.384325  5885 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:46.384795  5885 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:46.385617  5885 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.386773  5885 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:46.392905  5714 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:14:46.279819  5955 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.397274  5955 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:46.320948  5501 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:14:46.400847  5891 consensus_queue.cc:1243] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer 597687e464174d1ea6043161e037d240 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:14:46.357579  6046 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:14:46.409924  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:14:46.429191   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 114620us
I20260709 13:14:46.440953  6034 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.415253  5963 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:46.441565  6034 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.450686   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:46.451362   846 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 10.
I20260709 13:14:46.421595  6010 consensus_queue.cc:1059] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:14:46.452620   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:46.469933   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 13 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 14 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.468184  5980 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:46.466692  6034 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:14:46.485946   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 135169us
W20260709 13:14:46.498005   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 176374us
I20260709 13:14:46.471592  5955 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:14:46.484901  5980 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.524955   996 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:46.525648   996 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:46.526580   996 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.529973  6015 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:46.530582  6015 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:14:46.530848   930 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:46.534061   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:46.537626  5980 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.540907   996 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:46.544816   751 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 5, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 5 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.511373  5955 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.595285   926 raft_consensus.cc:1240] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:46.531492  6015 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.603235   751 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 4 to 7, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 7 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:46.608227   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 103724us
I20260709 13:14:46.609489  1593 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:46.627251   930 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:14:46.627823  5955 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:46.607780   751 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 9 to 10, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:46.632463  6015 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.604055  6032 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:14:46.648916   996 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:14:46.651207   996 pending_rounds.cc:85] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 9
I20260709 13:14:46.653040   996 pending_rounds.cc:107] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 13.10
W20260709 13:14:46.649380   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 144822us
I20260709 13:14:46.653688   996 raft_consensus.cc:2889] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:46.658756   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 72415.3us
W20260709 13:14:46.659485  5891 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:14:46.645946  5840 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:46.686303  5774 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:46.696233   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:46.703230   852 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:14:46.703366   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47696.6us
I20260709 13:14:46.701442  5267 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.722445  5267 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.724630  5267 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:46.725296  5840 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:46.725871  5840 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:46.752501   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:46.753355   929 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:46.763018  5840 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:46.770787   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:46.782823   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:46.814880   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:46.820253  5980 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.821813   995 raft_consensus.cc:2368] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11.
I20260709 13:14:46.823148  5980 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:46.810035  6070 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:46.790953  5947 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:14:46.871069   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:46.891306  6070 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.891676   847 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.891825  6070 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:46.900259   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 168892us
I20260709 13:14:46.912811   995 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:46.913201   995 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:46.913872   995 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:46.916115   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 88967.7us
I20260709 13:14:46.916934  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:14:46.910676  6070 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.921124   968 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:46.925956  1600 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:14:46.926369  1600 pending_rounds.cc:85] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 9
I20260709 13:14:46.926565  1600 pending_rounds.cc:107] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:14:46.926769  1600 raft_consensus.cc:2889] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:46.929471   847 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:46.930444   896 leader_election.cc:400] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11.
I20260709 13:14:46.930780   896 leader_election.cc:403] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:46.931433   924 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:46.930627   969 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:46.932674  5886 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:14:46.936843  6063 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:46.937793  6063 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:46.941768   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:46.942471   846 raft_consensus.cc:2368] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11.
I20260709 13:14:46.944267   853 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:46.950912  6063 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.952518  6030 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.953023  6030 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:46.955101  6030 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:46.960557   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:46.944270  5730 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.936374  5983 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:14:46.932099  5980 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.993391  5730 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) with higher term. Message: Invalid argument: T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11.
I20260709 13:14:47.043962 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:47.086218   895 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.087179   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:47.087795   924 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.092648  5991 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:47.101027   895 leader_election.cc:400] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) with higher term. Message: Invalid argument: T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for earlier term 10. Current term is 11.
I20260709 13:14:47.103837  6084 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:47.111235   926 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:47.131791   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:47.127954   851 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 7 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.123425  5982 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 7 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.176242   924 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 16.
I20260709 13:14:47.179767  1593 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:47.181794   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:47.187322   995 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:47.200145   930 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:14:47.206696   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:47.176774  6070 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:47.216320  5998 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:47.218083  1600 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 7 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.211447   895 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.224442  5998 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
W20260709 13:14:47.227315   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 56805.1us
I20260709 13:14:47.228710  6034 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:47.241279  5998 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.272586   995 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:47.280619   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:47.286197  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:47.281467   998 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:47.312829   929 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:47.317569  5981 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:14:47.334506  5981 consensus_queue.cc:1243] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:47.339234   995 pending_rounds.cc:85] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Aborting all ops after (but not including) 12
I20260709 13:14:47.339558   995 pending_rounds.cc:107] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:14:47.339818   995 raft_consensus.cc:2889] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:47.293548  5302 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:47.373929  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:47.377020  1004 raft_consensus.cc:606] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.386531  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:47.387377  1004 raft_consensus.cc:606] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.393500   929 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 7 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.407750  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:47.408362  1004 raft_consensus.cc:606] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.408676  1004 raft_consensus.cc:614] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:47.414507   853 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:47.420394  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:47.421079  1004 raft_consensus.cc:606] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:47.435861  1004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:47.437002  1004 raft_consensus.cc:606] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:47.437430  1004 raft_consensus.cc:614] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:47.420082  6034 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:14:47.442998 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:47.444324 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:47.458616  6051 raft_consensus.cc:993] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.459137  5963 raft_consensus.cc:1081] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.460789   855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:47.461290   855 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:47.461655   855 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.464040   929 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:47.473899  6073 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:14:47.495872   855 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.520885  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:47.521704  1004 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:47.522099  1004 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:47.523100  1004 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.524389  1004 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.533556   855 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:47.533736  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:47.536113  1600 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.542613  1600 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:47.543705   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:47.544523  6060 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:47.544999  6060 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:47.545917  6060 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.546280  1004 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
W20260709 13:14:47.567310  6091 log.cc:927] Time spent T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:14:47.614780   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 233) took 11030 ms. Trace:
I20260709 13:14:47.615844   750 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 17, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 17 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.617398   750 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.618786   750 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.620378   750 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.621843   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.623270   750 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 11, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.615245   751 rpcz_store.cc:276] 0709 13:14:36.584077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.584289 (+   212us) service_pool.cc:224] Handling call
0709 13:14:47.614689 (+11030400us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:46.764260 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:46.764938 (+   678us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:46.764993 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:14:46.765015 (+    22us) tablet.cc:662] Decoding operations
0709 13:14:46.766073 (+  1058us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:46.766167 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:46.766197 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:14:46.767020 (+   823us) tablet.cc:701] Row locks acquired
0709 13:14:46.767071 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:14:46.767232 (+   161us) write_op.cc:270] Start()
0709 13:14:46.767386 (+   154us) write_op.cc:276] Timestamp: P: 1783602886767165 usec, L: 0
0709 13:14:46.767407 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:14:46.768766 (+  1359us) log.cc:844] Serialized 3599 byte log entry
0709 13:14:47.092058 (+323292us) op_driver.cc:464] REPLICATION: finished
0709 13:14:47.209521 (+117463us) write_op.cc:301] APPLY: starting
0709 13:14:47.209604 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:47.297461 (+ 87857us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:47.297553 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:47.450878 (+153325us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:47.451101 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:14:47.451160 (+    59us) write_op.cc:312] APPLY: finished
0709 13:14:47.456697 (+  5537us) log.cc:844] Serialized 89 byte log entry
0709 13:14:47.458103 (+  1406us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.458222 (+   119us) write_op.cc:454] Released schema lock
0709 13:14:47.458271 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:47.480090 (+ 21819us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.480832 (+   742us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":612706,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":117226,"apply.run_cpu_time_us":14773,"apply.run_wall_time_us":263637,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":54499,"prepare.run_cpu_time_us":30113,"prepare.run_wall_time_us":127933,"raft.queue_time_us":119415,"raft.run_cpu_time_us":37012,"raft.run_wall_time_us":45183,"replication_time_us":324504,"thread_start_us":214929,"threads_started":4,"wal-append.queue_time_us":158904}]]}
I20260709 13:14:47.630102   750 catalog_manager.cc:5697] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.634580  5982 raft_consensus.cc:993] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.635095  6034 raft_consensus.cc:1081] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.636694   855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:47.637238   855 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:47.637567   855 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.645493   855 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.633648   750 catalog_manager.cc:5697] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.648530   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:47.649101   928 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.649448   750 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.651137   750 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 13 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.653061   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:47.656080   995 raft_consensus.cc:3055] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:47.656383   995 raft_consensus.cc:740] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:47.657189   995 consensus_queue.cc:260] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.658674   995 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.655042  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:47.661453  1600 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.674820  1600 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:47.655483   855 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:47.675805   995 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:47.676429   750 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 7 to 10, term changed from 12 to 14, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.691954   821 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:47.699321  6060 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:47.708513   928 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 15.
I20260709 13:14:47.708828   750 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 14, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 14 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.710489   750 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 13 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.715559  5302 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:47.717270  5302 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.719142   750 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.720853   750 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 16, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.724074   750 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.725266   750 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.726364   750 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.727273   750 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.710829  6060 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:47.729076   750 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 14, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.731099   750 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.732303   750 catalog_manager.cc:5697] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.733511   750 catalog_manager.cc:5697] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 7 to 9, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:47.734385   750 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 16, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.735343   750 catalog_manager.cc:5697] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.736585   750 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:47.738255   750 catalog_manager.cc:5697] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 8 to 10, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 10 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.739493   750 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.752183   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:47.752835   851 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:47.761549   929 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:14:47.761755   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:14:47.762589   995 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:47.758916  5302 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:47.765621   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:14:47.770730  5886 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:47.771145  5886 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:14:47.771498  5886 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.773221  6070 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:47.744352  6060 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:47.810648  5982 raft_consensus.cc:993] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.811213  5982 raft_consensus.cc:1081] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:47.812863   851 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:47.813381   851 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:47.813709   851 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.816192  5886 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.818337  5886 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:47.820182   851 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.821938   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:47.828078   995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:47.829108   995 raft_consensus.cc:2393] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 14: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:14:47.832124   855 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:14:47.834203   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:47.835852  6030 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:47.836778  6030 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:47.837882  6030 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.840832   996 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.845278  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:47.845932  1004 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:47.846205  1004 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:47.847121  1004 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:47.848652  1004 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.852394   996 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:14:47.859514  6062 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:47.860915   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:47.861688   926 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.863274   851 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:47.855022  1004 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:47.864823   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:47.891007   926 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:47.891775  6060 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:47.895664  6060 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:47.925032  6060 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:48.019230  1600 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:48.020551  6060 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:48.047025  6071 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.047591  6071 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:48.052237   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829" is_pre_election: true
I20260709 13:14:48.052750   851 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:48.057945  6071 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:48.059108   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:14:48.060143   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:48.061901   996 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:48.062743  6071 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:48.063112  6071 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.063431  6071 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.071295  6060 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:48.053539   749 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 6 to 9, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 9 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:48.080355   740 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 85bbb1b886474a538a2f1a32dd91b560 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3bb5726887244caca146e176cc6bd070 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:48.087481  6071 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:48.090338   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:48.090854   855 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.086449   749 catalog_manager.cc:5697] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 12 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 13 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.097545   749 catalog_manager.cc:5697] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.099411   855 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:48.100577   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:48.102100  6034 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:48.102957  6034 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:48.103927  6034 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:48.122396  6071 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:48.124217   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:48.207820   749 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 122) took 5429 ms. Trace:
I20260709 13:14:48.215776   749 rpcz_store.cc:276] 0709 13:14:42.779040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.779445 (+   405us) service_pool.cc:224] Handling call
0709 13:14:48.207761 (+5428316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:48.109939 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:48.110521 (+   582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:48.110558 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:14:48.110579 (+    21us) tablet.cc:662] Decoding operations
0709 13:14:48.111169 (+   590us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:48.111232 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:48.111257 (+    25us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:14:48.111554 (+   297us) tablet.cc:701] Row locks acquired
0709 13:14:48.111577 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:14:48.111732 (+   155us) write_op.cc:270] Start()
0709 13:14:48.111849 (+   117us) write_op.cc:276] Timestamp: P: 1783602888111689 usec, L: 0
0709 13:14:48.111872 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:14:48.125638 (+ 13766us) log.cc:844] Serialized 1293 byte log entry
0709 13:14:48.127508 (+  1870us) op_driver.cc:464] REPLICATION: finished
0709 13:14:48.127993 (+   485us) write_op.cc:301] APPLY: starting
0709 13:14:48.128072 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:48.129900 (+  1828us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:48.130007 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:48.143135 (+ 13128us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:48.143327 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:14:48.143380 (+    53us) write_op.cc:312] APPLY: finished
0709 13:14:48.143631 (+   251us) log.cc:844] Serialized 41 byte log entry
0709 13:14:48.147082 (+  3451us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.147191 (+   109us) write_op.cc:454] Released schema lock
0709 13:14:48.147238 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:48.147684 (+   446us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.147717 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4623,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":269,"apply.run_cpu_time_us":5308,"apply.run_wall_time_us":19612,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12540,"num_ops":3,"prepare.queue_time_us":123,"prepare.run_cpu_time_us":3101,"prepare.run_wall_time_us":16036,"replication_time_us":15493}]]}
I20260709 13:14:48.232353  1599 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:48.235637  6070 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:14:48.236204   751 catalog_manager.cc:5697] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:48.243206   751 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 17, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 17 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:48.256444   751 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:48.268168   751 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 10 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 14 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.272229   751 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 9 to 11, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.281493   996 raft_consensus.cc:1240] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:48.283265  6030 consensus_queue.cc:1059] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:48.284073  6030 consensus_queue.cc:1243] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer 3bb5726887244caca146e176cc6bd070 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:48.284539  6030 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:48.284881  6030 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:48.285754  6030 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:48.287036  6030 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.310837   751 catalog_manager.cc:5697] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.317203   926 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:48.325218   996 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:48.328120  5945 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:48.345785  6062 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:48.402863  1004 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:48.405114  6060 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:48.445340 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:48.479383   750 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:48.507117   930 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:48.511541  6060 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:48.531924   749 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 14 to 15, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 15 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:48.588011   750 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 15, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 15 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.590039   750 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 13 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } }
I20260709 13:14:48.616951   851 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:14:48.618362  6034 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:48.635277   753 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55304 (request call id 229) took 5292 ms. Trace:
I20260709 13:14:48.635859   753 rpcz_store.cc:276] 0709 13:14:43.343172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.343389 (+   217us) service_pool.cc:224] Handling call
0709 13:14:43.678010 (+334621us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000daf54. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa9090 00007f0d02b47f3a 00007f0d088fe1cc 00007f0d089fc37e 00007f0d06ebd2a4 00007f0d06ebd232 00007f0d06ebd161 00007f0d06ebd0dc 00007f0d06ebc392 00007f0d00e7a48c 00007f0d00e798c6
0709 13:14:48.635092 (+4957082us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30455,"reported_tablets":49,"spinlock_wait_cycles":377354112}
I20260709 13:14:48.810173   926 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:14:48.810832   926 pending_rounds.cc:85] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Aborting all ops after (but not including) 9
I20260709 13:14:48.811203   926 pending_rounds.cc:107] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:14:48.811620   926 raft_consensus.cc:2889] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:48.881582  5891 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:49.005721   851 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:49.014359  1600 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:49.020403  5946 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:14:49.037480  5945 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 10 to 12, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:49.052245  1600 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 10 to 12, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:49.047699   996 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:49.057101  6060 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:49.062572   996 raft_consensus.cc:2955] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 10 to 12, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:49.096462   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 85bbb1b886474a538a2f1a32dd91b560 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260709 13:14:49.184193   753 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 10 to 12, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 15 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:49.242549   739 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 85bbb1b886474a538a2f1a32dd91b560 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 10 but the committed config has opid_index of 12
I20260709 13:14:49.295787   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 85bbb1b886474a538a2f1a32dd91b560 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:42376
I20260709 13:14:49.342388  6118 tablet_replica.cc:333] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:14:49.343755  6118 raft_consensus.cc:2243] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:49.344841  6118 raft_consensus.cc:2272] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:49.359912  6118 ts_tablet_manager.cc:1916] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:49.404765  6118 ts_tablet_manager.cc:1929] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:14:49.407940  6118 log.cc:1199] T 85bbb1b886474a538a2f1a32dd91b560 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/85bbb1b886474a538a2f1a32dd91b560
I20260709 13:14:49.730425 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:49.796106   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 55954.2us
I20260709 13:14:49.827216   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 85bbb1b886474a538a2f1a32dd91b560 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
W20260709 13:14:49.845436   737 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 118705us
W20260709 13:14:49.938836   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 120808us
W20260709 13:14:50.087723   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 51755.5us
W20260709 13:14:50.135557   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 200776us
I20260709 13:14:50.136762   855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:50.137580   855 raft_consensus.cc:606] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
W20260709 13:14:50.135700   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 169948us
W20260709 13:14:50.139906   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 202209us
W20260709 13:14:50.148459   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 272432us
W20260709 13:14:50.179589  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 134802us
W20260709 13:14:50.187352   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 217703us
W20260709 13:14:50.196017   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 57542us
I20260709 13:14:50.198155  6120 raft_consensus.cc:993] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
W20260709 13:14:50.219359   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 80420.1us
I20260709 13:14:50.224987  6060 raft_consensus.cc:1081] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.237066   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:50.237704   847 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:50.246882   855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:50.247493   855 raft_consensus.cc:606] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:50.276026   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:50.276609   847 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:50.307248   998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:50.318142   998 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.318550   998 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.328277  6119 raft_consensus.cc:993] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:50.328778  6119 raft_consensus.cc:1081] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:50.292150   851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:50.336632   851 raft_consensus.cc:606] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:50.342743 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:50.344488 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:50.339452   998 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.367389   998 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:50.368003   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:50.368891   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:50.380808   847 raft_consensus.cc:3055] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:50.380847   928 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.381409   847 raft_consensus.cc:740] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:50.390466   847 consensus_queue.cc:260] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.391640   847 raft_consensus.cc:3060] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.381354  6131 raft_consensus.cc:993] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.396224  6119 raft_consensus.cc:1081] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.400609   847 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 17.
I20260709 13:14:50.389271   928 raft_consensus.cc:2468] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 17.
I20260709 13:14:50.402016   969 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:50.403111  6123 raft_consensus.cc:2804] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:50.403704  6123 raft_consensus.cc:697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:50.404775  6123 consensus_queue.cc:237] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.407855  6131 raft_consensus.cc:993] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.408773  6119 raft_consensus.cc:1081] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.426020   749 catalog_manager.cc:5697] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 16 to 17, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 17 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:50.430872   928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42390
I20260709 13:14:50.432700  6114 raft_consensus.cc:993] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.433663  6114 raft_consensus.cc:1081] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:50.431679   928 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.435927   928 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.458859   995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:50.459547   995 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.459957   995 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.461169   998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:50.461756   998 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.462054   998 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.475234   998 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:50.479534  1004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:50.480204  1004 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:50.480557  1004 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:50.480679   928 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.482019   998 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:50.482153  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:50.482784  1600 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.492120   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:50.492846   847 raft_consensus.cc:3055] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:50.504264   847 raft_consensus.cc:740] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:50.506116  1004 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.505088  1600 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:50.507530   847 consensus_queue.cc:260] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:50.509121   855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:50.509209   968 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:14:50.509752   855 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:50.509754   847 raft_consensus.cc:3060] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.510118   855 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:50.511302  6123 raft_consensus.cc:2804] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:50.511072   855 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.512046  6123 raft_consensus.cc:697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:50.512310   855 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:50.512740  1004 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 18 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:50.513036  6123 consensus_queue.cc:237] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:50.514206   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:50.514746   925 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:50.525396   925 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 18.
I20260709 13:14:50.525974   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:50.526605   851 raft_consensus.cc:3055] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:50.526901   851 raft_consensus.cc:740] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:50.505455   995 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.528240   967 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:50.528512   851 consensus_queue.cc:260] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.520712   855 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 18.
I20260709 13:14:50.529913   928 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:50.530035   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:50.530660   846 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:50.530977   846 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:50.531849  6154 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:50.531688   846 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.532338  6154 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:50.532796   846 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.533140  6154 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.534737   847 raft_consensus.cc:2468] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:14:50.538391   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:50.540437   995 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:50.541115   846 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:50.538918   998 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.542385   969 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:50.543566   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:50.544168   928 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.547325   851 raft_consensus.cc:3060] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.545872  6152 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:50.551013  6152 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:50.554457   998 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:14:50.556267   928 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:14:50.555971  6152 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.558410   896 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:50.560155  6144 raft_consensus.cc:2804] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:50.560690  6144 raft_consensus.cc:697] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:50.561573  6144 consensus_queue.cc:237] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:50.569666   851 raft_consensus.cc:2468] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:14:50.729909   750 catalog_manager.cc:5697] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 15 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.845835   926 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:50.848940   846 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:50.853993  5891 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:50.865455   851 raft_consensus.cc:1275] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:50.869124  6152 consensus_queue.cc:1048] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:50.872789   749 catalog_manager.cc:5697] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 15 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.882835   749 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 17 to 18, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 18 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:50.892808   749 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:50.932639   819 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ec15122641447a3bd48017eb990578d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.936599  1004 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:50.938163  6136 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:50.964310  1600 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:50.966169  6130 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:50.971276  6062 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:50.973228  1003 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:50.978106   851 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 18 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:50.998533  1004 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:51.001032  6130 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:51.010120  1003 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:51.011781  6030 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:51.017989   749 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:50.984965   925 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:51.034626  5946 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:51.069103   851 raft_consensus.cc:1275] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:51.084129  6159 consensus_queue.cc:1048] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:51.091622   926 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:51.093444   928 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:14:51.132808  6180 log.cc:927] Time spent T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:14:51.190084   851 raft_consensus.cc:1275] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:51.189797  6148 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:14:51.267114  6148 consensus_queue.cc:1048] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:51.278676  6189 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:51.318993   925 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.345395 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:51.384455   968 consensus_peers.cc:597] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c72e2f13b3404583a1c34e15098c42b9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.442802   851 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:51.451617   851 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:14:51.454160   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50881.9us
W20260709 13:14:51.555895   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 131134us
I20260709 13:14:51.566650   930 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:14:51.572274   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 152281us
I20260709 13:14:51.590761  6152 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:14:51.610011  6212 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:51.612012  6208 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:51.729549  6222 ts_tablet_manager.cc:933] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:51.731964  6123 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.747087   847 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.818400   998 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:51.822391   930 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.836637  6222 tablet_copy_client.cc:323] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:14:51.839303   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet 2ec15122641447a3bd48017eb990578d from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:14:51.839906   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet 2ec15122641447a3bd48017eb990578d from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-2ec15122641447a3bd48017eb990578d
I20260709 13:14:51.847641   847 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:51.850131  6152 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:51.858982  6152 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.860778   865 tablet_copy_source_session.cc:215] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:14:51.863361   968 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 701b27559f8e47f9b63e5d220d3d5a58. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.862134   846 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.872476  6222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec15122641447a3bd48017eb990578d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:51.874132   925 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:51.877038   925 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:51.882767  6148 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:51.901737   930 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:51.909698   750 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:51.912099   894 consensus_peers.cc:597] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ee81d93e84c4c689b02deb555210ba4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.911952   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.911351  6221 ts_tablet_manager.cc:933] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:51.925269  1001 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:51.930835   846 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:51.943905  6221 tablet_copy_client.cc:323] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:14:51.952055  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet c72e2f13b3404583a1c34e15098c42b9 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:14:51.952708  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet c72e2f13b3404583a1c34e15098c42b9 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-c72e2f13b3404583a1c34e15098c42b9
I20260709 13:14:51.958047  6144 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:14:51.959715  6121 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:52.014232  1014 tablet_copy_source_session.cc:215] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.018507  6221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72e2f13b3404583a1c34e15098c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.091991   847 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:52.073333   998 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:52.071312  6084 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:52.130234  6221 tablet_copy_client.cc:806] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.130932  6221 tablet_copy_client.cc:670] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.139727  6221 tablet_copy_client.cc:538] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.160544  6221 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:52.195205  6222 tablet_copy_client.cc:806] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.196023  6222 tablet_copy_client.cc:670] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.204499  6222 tablet_copy_client.cc:538] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.239049   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
I20260709 13:14:52.244138   998 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership
I20260709 13:14:52.250589   753 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 12, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:52.257231   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
I20260709 13:14:52.257875   998 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership
W20260709 13:14:52.263149   740 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 701b27559f8e47f9b63e5d220d3d5a58 already in progress [suppressed 3 similar messages]
I20260709 13:14:52.323076  6222 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:52.328506  5891 raft_consensus.cc:993] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:52.329067  5891 raft_consensus.cc:1081] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:52.332422   847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:52.332971   847 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.333313   847 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.360195  6233 ts_tablet_manager.cc:933] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:14:52.371202   740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 108079us
I20260709 13:14:52.377220   847 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.412191   847 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:52.412860   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:52.413641   930 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.419061  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
I20260709 13:14:52.419672  1001 raft_consensus.cc:606] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership
W20260709 13:14:52.432560   801 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:14:52.433024   801 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 701b27559f8e47f9b63e5d220d3d5a58 already in progress
W20260709 13:14:52.433293   801 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 701b27559f8e47f9b63e5d220d3d5a58 already in progress
I20260709 13:14:52.441572   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:52.443975   998 raft_consensus.cc:3055] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:52.444756   998 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:52.450874  6233 tablet_copy_client.cc:323] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:14:52.452363   930 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:52.452973   998 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.454979   998 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.454453   818 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:52.456259  6149 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:52.456729  6149 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:52.457567  6149 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.468372   998 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:14:52.469897  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 701b27559f8e47f9b63e5d220d3d5a58 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:14:52.470445  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 701b27559f8e47f9b63e5d220d3d5a58 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-701b27559f8e47f9b63e5d220d3d5a58
I20260709 13:14:52.480824  1014 tablet_copy_source_session.cc:215] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.480742   750 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 13 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:52.484342  6233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701b27559f8e47f9b63e5d220d3d5a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.490490 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:52.557482  6236 ts_tablet_manager.cc:933] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:52.562906  6233 tablet_copy_client.cc:806] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.563586  6233 tablet_copy_client.cc:670] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.587296  6236 tablet_copy_client.cc:323] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:14:52.589361   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 9ee81d93e84c4c689b02deb555210ba4 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:14:52.589943   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 9ee81d93e84c4c689b02deb555210ba4 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-9ee81d93e84c4c689b02deb555210ba4
I20260709 13:14:52.602059   940 tablet_copy_source_session.cc:215] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.607237   847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:52.607905  6233 tablet_copy_client.cc:538] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.610002   847 raft_consensus.cc:606] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:14:52.614611  6236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee81d93e84c4c689b02deb555210ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.617974   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:52.618567   998 raft_consensus.cc:606] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:52.619694 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:52.621204 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:52.650125  6221 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.651264  6221 tablet_bootstrap.cc:492] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:52.652067  6221 ts_tablet_manager.cc:1403] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.492s	user 0.053s	sys 0.024s
I20260709 13:14:52.678982  6236 tablet_copy_client.cc:806] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.679672  6236 tablet_copy_client.cc:670] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.682261  6221 raft_consensus.cc:359] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.683125  6221 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:52.686281  6233 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:52.701520  6236 tablet_copy_client.cc:538] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.705679  6221 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.720578  6222 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=3 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.723747  6222 tablet_bootstrap.cc:492] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:52.744982  6222 ts_tablet_manager.cc:1403] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.422s	user 0.037s	sys 0.010s
I20260709 13:14:52.747169  6221 ts_tablet_manager.cc:1434] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.000s
I20260709 13:14:52.748807  6190 raft_consensus.cc:993] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:52.749392  6190 raft_consensus.cc:1081] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:52.751029   925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42382
I20260709 13:14:52.751543   925 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.751883   925 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.759373   925 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.762367   847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:52.762890   847 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.775233  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-c72e2f13b3404583a1c34e15098c42b9 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:14:52.775823  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-c72e2f13b3404583a1c34e15098c42b9 on tablet c72e2f13b3404583a1c34e15098c42b9 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:52.781433   847 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:14:52.783118   895 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:52.784560  6222 raft_consensus.cc:359] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.788434  6222 raft_consensus.cc:740] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:52.789255  6084 raft_consensus.cc:2804] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:52.789105  6222 consensus_queue.cc:260] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:52.797390   925 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:52.798364   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:52.798975   998 raft_consensus.cc:3055] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:52.801131   998 raft_consensus.cc:740] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:52.801873   998 consensus_queue.cc:260] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.802892   998 raft_consensus.cc:3060] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.798704  6084 raft_consensus.cc:697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:52.805943  6084 consensus_queue.cc:237] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.817719  6149 raft_consensus.cc:993] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:52.818315  6199 raft_consensus.cc:1081] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:52.817699  6222 ts_tablet_manager.cc:1434] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.072s	user 0.000s	sys 0.007s
I20260709 13:14:52.820307  1003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:52.822081  1003 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.822451  1003 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.829838  1003 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.833521   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-2ec15122641447a3bd48017eb990578d received from {username='slave'} at 127.0.0.1:43438
I20260709 13:14:52.834139   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-2ec15122641447a3bd48017eb990578d on tablet 2ec15122641447a3bd48017eb990578d with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:52.836587  1003 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:52.837145   851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:52.847930   851 raft_consensus.cc:3055] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:52.848292   851 raft_consensus.cc:740] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:52.848980   851 consensus_queue.cc:260] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.850184   851 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.857676   930 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:52.859644  6199 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:52.839025   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:52.819950   998 raft_consensus.cc:2468] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:14:52.872802   925 raft_consensus.cc:3060] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.821305  6236 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:52.898382   998 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:14:52.899443   819 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:52.905819  6193 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:52.924422   851 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:52.929714   969 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:52.930642  6148 raft_consensus.cc:2804] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:52.931244  6148 raft_consensus.cc:697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:52.932837  6148 consensus_queue.cc:237] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:52.953661   925 raft_consensus.cc:2468] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:53.077723   749 catalog_manager.cc:5697] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 15 to 16, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:53.244560   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 109497us
W20260709 13:14:53.297901   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52263.8us
I20260709 13:14:53.335887   750 catalog_manager.cc:5697] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 14, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.344334   996 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:53.428603  1600 raft_consensus.cc:1217] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:14:53.492734  1598 raft_consensus.cc:1217] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
W20260709 13:14:53.505949  6236 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:14:53.506115   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 73947.4us
W20260709 13:14:53.562121   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 128201us
I20260709 13:14:53.569211   846 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:53.572659  6260 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:53.577533  6255 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:14:53.584147   926 raft_consensus.cc:1275] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:53.585783  6233 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:53.593505  6233 tablet_bootstrap.cc:492] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:53.594619  6233 ts_tablet_manager.cc:1403] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.909s	user 0.044s	sys 0.015s
W20260709 13:14:53.600771   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 44230us
I20260709 13:14:53.607852  6233 raft_consensus.cc:359] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:53.610524  6233 raft_consensus.cc:740] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:53.611418  6233 consensus_queue.cc:260] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:53.621784  6233 ts_tablet_manager.cc:1434] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 13:14:53.625599  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-701b27559f8e47f9b63e5d220d3d5a58 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:14:53.626235  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-701b27559f8e47f9b63e5d220d3d5a58 on tablet 701b27559f8e47f9b63e5d220d3d5a58 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:53.631062  6260 consensus_queue.cc:1048] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:53.698341 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:53.805063   852 raft_consensus.cc:1275] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:53.875393  6199 raft_consensus.cc:1064] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
W20260709 13:14:53.915436   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 136041us
I20260709 13:14:53.878875  6199 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:53.949008  6236 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:54.040486  6262 consensus_queue.cc:1048] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
W20260709 13:14:54.046953   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68376.5us
I20260709 13:14:54.045569  6236 tablet_bootstrap.cc:492] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:54.054313  6236 ts_tablet_manager.cc:1403] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 1.233s	user 0.049s	sys 0.013s
I20260709 13:14:54.071869  6190 raft_consensus.cc:1064] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
W20260709 13:14:54.108603   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 57488.9us
I20260709 13:14:54.115465  6190 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:54.121155   819 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.073168  6236 raft_consensus.cc:359] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:54.124735  6236 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
W20260709 13:14:54.132731   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 153196us
W20260709 13:14:54.160652   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44169.1us
I20260709 13:14:54.163874  6236 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:54.166256  6236 ts_tablet_manager.cc:1434] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.001s
W20260709 13:14:54.169577   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 45262us
I20260709 13:14:54.196481   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-9ee81d93e84c4c689b02deb555210ba4 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:14:54.197043   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-9ee81d93e84c4c689b02deb555210ba4 on tablet 9ee81d93e84c4c689b02deb555210ba4 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:54.390336  1600 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:14:54.396334   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 365300us
I20260709 13:14:54.419252  6149 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:14:54.440471  1600 raft_consensus.cc:1217] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.11->[16.12-16.12]   Dedup: 16.12->[]
W20260709 13:14:54.452708  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 119048us
W20260709 13:14:54.454730  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64662.6us
W20260709 13:14:54.455684   970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.469197  1600 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:54.474334  6190 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:14:54.536448  6304 log.cc:927] Time spent T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:14:54.567749  1003 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:54.572743   852 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:54.596369  6246 raft_consensus.cc:1064] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:14:54.607052  6246 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:54.610106   924 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:14:54.651852   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 110596us
I20260709 13:14:54.664702  6299 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:54.669248  6281 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:54.700871   925 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:14:54.756757   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 252341us
I20260709 13:14:54.795965  1599 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:14:54.803750   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 51197.9us
I20260709 13:14:54.814675  6200 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:54.820219  6313 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:54.827512   922 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:14:54.850853   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54549.1us
I20260709 13:14:54.863925  6315 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:54.867063  1003 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:54.867786  6144 raft_consensus.cc:1064] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:14:54.875550  6299 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:54.884330  1592 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:54.887072   852 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:54.888338   927 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:54.875061  6144 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:54.925693  6259 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:14:54.957103  6250 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:14:55.034988   752 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.050906  1599 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:55.056246  6263 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:55.079196   852 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:55.081306  6121 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:55.082597  6314 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 10 to 12, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.084968  6259 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.087779   997 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 10 to 12, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.095983   927 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.134181   996 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:55.155033  6144 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:14:55.188973  1002 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.191569   930 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 10 to 12, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.225409  1592 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 10 to 12, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:55.243623  6322 log.cc:927] Time spent T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:14:55.261907  6144 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.266642  1599 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.274081  1596 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.343760  1003 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.360570   852 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.428372  1003 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:55.437233   749 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 10 to 12, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.442327   749 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 12 to 13, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.443976   749 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.504462 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:55.509624  1596 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:55.520407  6281 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:55.522046   927 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:55.529157  6281 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:55.537060   927 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.531222  6268 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.548792  1596 raft_consensus.cc:2955] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.636473   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c72e2f13b3404583a1c34e15098c42b9 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:55.656340   927 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:55.680598  1596 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:14:55.682134  1003 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:14:55.684201  6121 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:55.684955   855 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:55.684980   852 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:55.694298   924 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:55.695685  6144 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:55.695953  1596 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:55.696437  1599 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:55.698310  6303 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:55.700139  6303 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:55.694386   930 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:55.705045  6259 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:55.705277  6283 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.707002  6259 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:55.719334  6303 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.725899  1599 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.727150   749 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.741950  1003 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.752848  6259 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.768324  1592 raft_consensus.cc:2955] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.777416   843 tablet_service.cc:1558] Processing DeleteTablet for tablet c72e2f13b3404583a1c34e15098c42b9 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39024
I20260709 13:14:55.794107   924 raft_consensus.cc:2955] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.803929   927 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.805060   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ec15122641447a3bd48017eb990578d with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:55.811712   749 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 13 to 14, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:55.820586   930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:14:55.821739   930 raft_consensus.cc:606] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:55.819410   752 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:55.832043   844 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ee81d93e84c4c689b02deb555210ba4 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:39024
I20260709 13:14:55.842567   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec15122641447a3bd48017eb990578d with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:51112
I20260709 13:14:55.851804   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9ee81d93e84c4c689b02deb555210ba4 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:55.828552  1592 raft_consensus.cc:2955] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:14:55.921710   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 45536.9us
I20260709 13:14:55.924787  1003 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:14:55.925463  1003 raft_consensus.cc:606] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:14:55.927291 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
W20260709 13:14:56.009302   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 77667.9us
I20260709 13:14:56.006318 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:14:56.016183   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 80948.3us
I20260709 13:14:56.032163  6329 tablet_replica.cc:333] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:14:56.038415   752 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 12 to 13, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:56.073463   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 701b27559f8e47f9b63e5d220d3d5a58 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:51112
I20260709 13:14:56.075230  6330 tablet_replica.cc:333] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:14:56.076436  6330 raft_consensus.cc:2243] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.077328  6330 raft_consensus.cc:2272] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.094902  6329 raft_consensus.cc:2243] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.096220  6329 raft_consensus.cc:2272] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.100068  6329 ts_tablet_manager.cc:1916] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:14:56.111750   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 188186us
I20260709 13:14:56.145745  6330 ts_tablet_manager.cc:1916] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:14:56.263835   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 83296us
W20260709 13:14:56.279767   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 121359us
I20260709 13:14:56.299546   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 701b27559f8e47f9b63e5d220d3d5a58 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:56.320127  6339 raft_consensus.cc:993] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:56.341210  6339 raft_consensus.cc:1081] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
W20260709 13:14:56.346616  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 45326.1us
I20260709 13:14:56.352934   852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46364
I20260709 13:14:56.353654   852 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:56.354038   852 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.369592   852 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:56.372495  6330 ts_tablet_manager.cc:1929] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:14:56.373086  6330 log.cc:1199] T 2ec15122641447a3bd48017eb990578d P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/2ec15122641447a3bd48017eb990578d
I20260709 13:14:56.374213  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:14:56.374832  1596 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.375407   852 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:14:56.381764  1596 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:56.386911   819 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:14:56.388777  6250 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:56.394690  6330 tablet_replica.cc:333] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:14:56.396620   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:56.397228   927 raft_consensus.cc:3055] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:56.397500   927 raft_consensus.cc:740] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:56.398222   927 consensus_queue.cc:260] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:56.399473   927 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.402740  6250 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:56.406351   927 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 12.
I20260709 13:14:56.406636  6250 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:56.422958   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 53883.9us
I20260709 13:14:56.428283   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 2ec15122641447a3bd48017eb990578d (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:14:56.457041  6299 raft_consensus.cc:993] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:56.457808  6299 raft_consensus.cc:1081] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:14:56.469314   852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:14:56.470115   852 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:56.470575   852 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.475898  6329 ts_tablet_manager.cc:1929] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
I20260709 13:14:56.476539  6329 log.cc:1199] T c72e2f13b3404583a1c34e15098c42b9 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/c72e2f13b3404583a1c34e15098c42b9
I20260709 13:14:56.478634  6329 tablet_replica.cc:333] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:14:56.500864   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet c72e2f13b3404583a1c34e15098c42b9 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:14:56.507016   852 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:56.516497  6329 raft_consensus.cc:2243] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.520776   852 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:56.523250  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:56.524446  1002 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:56.524798  1002 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:56.525585  1002 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:56.526856  1002 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.528569  6329 raft_consensus.cc:2272] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.532769   752 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 11 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.535585   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:56.536252   930 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.544570  6330 raft_consensus.cc:2243] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:56.545717  6330 raft_consensus.cc:2272] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:56.549122  6330 ts_tablet_manager.cc:1916] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:56.550861  1002 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:56.556680  6329 ts_tablet_manager.cc:1916] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:56.560907   930 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:14:56.562475   818 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:56.563803  6250 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:56.564354  6250 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:56.565281  6250 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:56.598574  6330 ts_tablet_manager.cc:1929] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.12
I20260709 13:14:56.599088  6330 log.cc:1199] T 701b27559f8e47f9b63e5d220d3d5a58 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/701b27559f8e47f9b63e5d220d3d5a58
I20260709 13:14:56.600600   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 701b27559f8e47f9b63e5d220d3d5a58 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:14:56.602838   752 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 12 to 13, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 13 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.633924  6329 ts_tablet_manager.cc:1929] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.13
I20260709 13:14:56.634557  6329 log.cc:1199] T 9ee81d93e84c4c689b02deb555210ba4 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/9ee81d93e84c4c689b02deb555210ba4
I20260709 13:14:56.636193   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 9ee81d93e84c4c689b02deb555210ba4 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:14:56.896847  1596 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:56.898859  6340 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:56.983178   927 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:56.985196  6316 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:57.014158 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:57.057654  1002 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:57.059428  6316 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:57.173384   922 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:57.179363  6314 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:14:57.367839   852 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:57.394515  1002 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:14:57.396224   819 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad6ca2a4a8134a10b60d7308b9a660db. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.398355  6314 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:57.400416   927 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:57.404251  6314 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:57.424577  6314 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.431384  1003 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.451642   927 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.461194   922 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:57.463120   752 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.483915  1002 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:57.483421   852 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:57.485816  6283 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:57.487016   894 consensus_peers.cc:597] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10b0600fc3f340eb999b12ed484fcdcb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.487546  6263 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:57.579891  6263 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.585642  1002 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.592103   852 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.658162   750 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.657526   852 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:57.698768  1002 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:57.705214  6340 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:57.713975   819 consensus_peers.cc:597] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfd817716cdc4448b3f884eecc94351c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.718535   927 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:57.720520  6340 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:57.742537  6303 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.749727   922 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.813400  6369 ts_tablet_manager.cc:933] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:57.815290   750 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:57.821650  1003 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.824247   852 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:14:57.852274   819 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dcb3d540c3fc49bf926540f7bd349110. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.852363   997 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:57.854125  6314 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:57.858536   922 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:57.860046  6303 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:57.884397  6340 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.884317  6369 tablet_copy_client.cc:323] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:14:57.905774   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet ad6ca2a4a8134a10b60d7308b9a660db from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:14:57.906325   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet ad6ca2a4a8134a10b60d7308b9a660db from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-ad6ca2a4a8134a10b60d7308b9a660db
I20260709 13:14:57.906213   930 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.916973   997 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:14:57.943907   753 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:57.944741   865 tablet_copy_source_session.cc:215] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:57.951092  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6ca2a4a8134a10b60d7308b9a660db. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:58.058398   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 99580.3us
I20260709 13:14:58.103453  6369 tablet_copy_client.cc:806] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.118824  6369 tablet_copy_client.cc:670] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
W20260709 13:14:58.136718   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66103.1us
I20260709 13:14:58.198025  6369 tablet_copy_client.cc:538] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.204082  6381 ts_tablet_manager.cc:933] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:58.213482  6387 ts_tablet_manager.cc:933] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:58.222576  6369 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:58.234372 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:58.249926   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 85965.7us
W20260709 13:14:58.310995   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 127288us
I20260709 13:14:58.357792  6381 tablet_copy_client.cc:323] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:14:58.375859   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 10b0600fc3f340eb999b12ed484fcdcb from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:14:58.376428   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 10b0600fc3f340eb999b12ed484fcdcb from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-10b0600fc3f340eb999b12ed484fcdcb
I20260709 13:14:58.386858  6387 tablet_copy_client.cc:323] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:14:58.388845   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet cfd817716cdc4448b3f884eecc94351c from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:14:58.389443   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet cfd817716cdc4448b3f884eecc94351c from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-cfd817716cdc4448b3f884eecc94351c
I20260709 13:14:58.398998   865 tablet_copy_source_session.cc:215] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.408798  6387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd817716cdc4448b3f884eecc94351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.442502   846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:58.444967   846 raft_consensus.cc:606] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
W20260709 13:14:58.443946   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 95737us
I20260709 13:14:58.440315   940 tablet_copy_source_session.cc:215] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:14:58.454062  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 91798.2us
I20260709 13:14:58.460635   852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:14:58.462757   852 raft_consensus.cc:606] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:14:58.484395  6381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b0600fc3f340eb999b12ed484fcdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.494158  6390 raft_consensus.cc:993] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:58.494741  6390 raft_consensus.cc:1081] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:14:58.508198  6381 tablet_copy_client.cc:806] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.509130  6381 tablet_copy_client.cc:670] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.513458  6381 tablet_copy_client.cc:538] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.516181 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:58.518235 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:58.523672  6381 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:58.526083  6387 tablet_copy_client.cc:806] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.526705  6387 tablet_copy_client.cc:670] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.529333   927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42390
I20260709 13:14:58.529951   927 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.530279   927 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.541421  6399 ts_tablet_manager.cc:933] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:14:58.543888  6375 raft_consensus.cc:993] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:58.544423  6375 raft_consensus.cc:1081] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:14:58.547233  1003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:14:58.547782  1003 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.548113  1003 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.551862  6387 tablet_copy_client.cc:538] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.559301   927 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.576936  1003 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.578557  6387 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:58.585135  6399 tablet_copy_client.cc:323] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:14:58.585777   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:58.586371   852 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:58.586668   852 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:58.587431   852 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.588537   852 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.590616   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet dcb3d540c3fc49bf926540f7bd349110 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:14:58.591130   927 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:14:58.591225   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet dcb3d540c3fc49bf926540f7bd349110 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-dcb3d540c3fc49bf926540f7bd349110
I20260709 13:14:58.585481  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:14:58.602598  1002 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.613516  1002 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:58.614076   852 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:14:58.615615  1003 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:14:58.614744   896 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:58.620618   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:14:58.621399   927 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.622534  6395 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:58.620605   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:14:58.623062  6395 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:14:58.623132   846 raft_consensus.cc:3055] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:58.623566   846 raft_consensus.cc:740] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:14:58.624033  6395 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.624279   846 consensus_queue.cc:260] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.625416   846 raft_consensus.cc:3060] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.625801   865 tablet_copy_source_session.cc:215] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.639500   927 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:58.640954   967 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:14:58.645624  6399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb3d540c3fc49bf926540f7bd349110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.651649  6376 raft_consensus.cc:2804] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:58.652434  6376 raft_consensus.cc:697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:14:58.653359  6376 consensus_queue.cc:237] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:14:58.713039   846 raft_consensus.cc:2468] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:14:58.742328   753 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 11 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.756868   749 catalog_manager.cc:5697] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 14, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.773563  6399 tablet_copy_client.cc:806] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.774190  6399 tablet_copy_client.cc:670] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.791738  6399 tablet_copy_client.cc:538] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.919106  6381 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:58.920253  6381 tablet_bootstrap.cc:492] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:58.921012  6381 ts_tablet_manager.cc:1403] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.398s	user 0.029s	sys 0.030s
I20260709 13:14:58.919248  6369 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=2 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:58.925441  6369 tablet_bootstrap.cc:492] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:58.925530  6381 raft_consensus.cc:359] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:58.926824  6381 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:58.927736  6381 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:58.926406  6369 ts_tablet_manager.cc:1403] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.704s	user 0.046s	sys 0.007s
I20260709 13:14:58.935019  6369 raft_consensus.cc:359] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:58.936216  6381 ts_tablet_manager.cc:1434] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:14:58.942451   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-10b0600fc3f340eb999b12ed484fcdcb received from {username='slave'} at 127.0.0.1:46362
I20260709 13:14:58.942883   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-10b0600fc3f340eb999b12ed484fcdcb on tablet 10b0600fc3f340eb999b12ed484fcdcb with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:58.946499  1599 raft_consensus.cc:1217] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 13:14:58.937431  6369 raft_consensus.cc:740] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:58.967352  6399 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:14:58.980486  6369 consensus_queue.cc:260] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:59.023651  6369 ts_tablet_manager.cc:1434] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.000s
I20260709 13:14:59.031137   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-ad6ca2a4a8134a10b60d7308b9a660db received from {username='slave'} at 127.0.0.1:43438
I20260709 13:14:59.031673   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-ad6ca2a4a8134a10b60d7308b9a660db on tablet ad6ca2a4a8134a10b60d7308b9a660db with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:59.047055  6387 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:59.048662  6387 tablet_bootstrap.cc:492] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:59.049388  6387 ts_tablet_manager.cc:1403] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.471s	user 0.040s	sys 0.007s
I20260709 13:14:59.055411  6387 raft_consensus.cc:359] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:59.056372  6387 raft_consensus.cc:740] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:59.057066  6387 consensus_queue.cc:260] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:59.069130  6387 ts_tablet_manager.cc:1434] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 13:14:59.084205   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-cfd817716cdc4448b3f884eecc94351c received from {username='slave'} at 127.0.0.1:43438
I20260709 13:14:59.084710   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-cfd817716cdc4448b3f884eecc94351c on tablet cfd817716cdc4448b3f884eecc94351c with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:59.099458   927 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:59.102772  6376 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:59.135341  1003 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:59.140157  6393 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:59.145049   852 raft_consensus.cc:1275] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:59.146651  6399 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:59.147046  6380 consensus_queue.cc:1048] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:59.154011  6399 tablet_bootstrap.cc:492] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:14:59.154922  6399 ts_tablet_manager.cc:1403] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.188s	user 0.027s	sys 0.013s
I20260709 13:14:59.176167   847 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:59.182192  6393 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:59.193084  6399 raft_consensus.cc:359] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:59.194185  6399 raft_consensus.cc:740] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:14:59.196734  6399 consensus_queue.cc:260] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:14:59.199216  1599 raft_consensus.cc:1217] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
W20260709 13:14:59.301088  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 98233.9us
I20260709 13:14:59.382555  6399 ts_tablet_manager.cc:1434] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.227s	user 0.008s	sys 0.000s
I20260709 13:14:59.524020 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:59.531992   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 47981.6us
I20260709 13:14:59.586642   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-dcb3d540c3fc49bf926540f7bd349110 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:14:59.587132   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-dcb3d540c3fc49bf926540f7bd349110 on tablet dcb3d540c3fc49bf926540f7bd349110 with peer 597687e464174d1ea6043161e037d240
I20260709 13:14:59.592808  6283 raft_consensus.cc:1064] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:14:59.595528  6283 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:14:59.616369   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 290547us
I20260709 13:14:59.648164  1592 raft_consensus.cc:1217] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 13:14:59.650739  1592 raft_consensus.cc:1217] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 13:14:59.702060  1592 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:59.704694  6431 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:59.714215   852 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:59.715771  6283 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:59.753975   847 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:59.750969  6283 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:59.768699  1002 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:59.775100   749 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:59.781230  6417 raft_consensus.cc:1064] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:14:59.784348  6417 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:14:59.805014  1592 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:59.859015  6423 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:59.890173  1599 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:59.891805  6397 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:59.899673  1003 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:14:59.903643  1002 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:59.904960   927 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:59.908212  6397 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:59.918417  6417 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:15:00.040532  6425 raft_consensus.cc:1064] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:00.042757  6425 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:00.073963  6396 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.081822   928 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:15:00.125959   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 122213us
I20260709 13:15:00.219894  6420 raft_consensus.cc:1064] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:00.241060  1003 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.223024  6420 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:00.281049   750 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.285494  1592 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.300964  1599 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:00.316437   928 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:00.336544  6391 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:00.359136  6430 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:00.359468  1002 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:00.364053  6426 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:00.373090   850 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:00.417665   928 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:00.426955  1592 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:00.439953  1592 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:00.448283   996 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:00.449815  6396 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:00.441097  6426 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:15:00.460920  6429 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:00.461043  6391 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:00.463893   930 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:00.491842  6426 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:00.518309   928 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:00.526571  1599 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:00.528934  6430 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.531505   996 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:00.538026  1599 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.539959  6448 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:00.539655  6420 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.543545   996 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.575875  6448 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:00.558429   930 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.595878   928 raft_consensus.cc:2955] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.606281  6449 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.614936  1002 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.627391  1592 raft_consensus.cc:2955] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.644021   752 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 10 to 11, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.651147   751 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.653630  6433 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.664528  1592 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.668823   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cfd817716cdc4448b3f884eecc94351c with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.664947  1599 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:15:00.701578   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 43964.3us
I20260709 13:15:00.702950   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 10b0600fc3f340eb999b12ed484fcdcb with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.704186   930 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.758102   994 tablet_service.cc:1558] Processing DeleteTablet for tablet cfd817716cdc4448b3f884eecc94351c with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:51112
I20260709 13:15:00.800659 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:00.817790   847 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:00.828092  1002 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:00.831174  6340 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:00.843056  6459 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.843042   753 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 8 to 9, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.851264  1002 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.862587   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad6ca2a4a8134a10b60d7308b9a660db with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.883823   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 10b0600fc3f340eb999b12ed484fcdcb with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:39024
I20260709 13:15:00.866503   752 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.895593  1599 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:00.906958  6462 tablet_replica.cc:333] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:00.908150  6462 raft_consensus.cc:2243] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:00.909978  6340 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:00.921957   753 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:00.928434  6463 tablet_replica.cc:333] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:00.929679  6463 raft_consensus.cc:2243] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:00.930424  6463 raft_consensus.cc:2272] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:00.932737  1599 raft_consensus.cc:2955] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.941599  1001 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:00.947283   996 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:00.961828   996 raft_consensus.cc:606] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:00.933903  6463 ts_tablet_manager.cc:1916] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:00.965623  6462 raft_consensus.cc:2272] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:00.966020   928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:15:00.966656   928 raft_consensus.cc:606] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:15:00.969156 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:00.994302  6462 ts_tablet_manager.cc:1916] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:01.004367 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:01.028097   919 tablet_service.cc:1558] Processing DeleteTablet for tablet ad6ca2a4a8134a10b60d7308b9a660db with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:42376
I20260709 13:15:01.030439  6463 ts_tablet_manager.cc:1929] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.8
I20260709 13:15:01.031036  6463 log.cc:1199] T 10b0600fc3f340eb999b12ed484fcdcb P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/10b0600fc3f340eb999b12ed484fcdcb
I20260709 13:15:01.034190   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 10b0600fc3f340eb999b12ed484fcdcb (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:01.042495  6421 raft_consensus.cc:993] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:15:01.043123  6421 raft_consensus.cc:1081] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:15:01.045374   850 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:15:01.045936   850 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:01.046380   850 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:01.055052   850 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.058099   850 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:01.056125  6464 tablet_replica.cc:333] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:01.059542   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:01.059587  6464 raft_consensus.cc:2243] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.060184   996 raft_consensus.cc:3055] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:01.060550   996 raft_consensus.cc:740] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:01.061065  6464 raft_consensus.cc:2272] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.061290   996 consensus_queue.cc:260] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.062485   996 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:01.065402  6449 raft_consensus.cc:993] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:01.065883  6449 raft_consensus.cc:1081] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:01.067966  6464 ts_tablet_manager.cc:1916] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:01.072597  1002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51144
I20260709 13:15:01.073174  1002 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:01.073555  1002 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:01.067955   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:01.077405   928 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:01.094823   850 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:01.102514  1599 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:01.102958  6464 ts_tablet_manager.cc:1929] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:15:01.103483  6464 log.cc:1199] T ad6ca2a4a8134a10b60d7308b9a660db P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/ad6ca2a4a8134a10b60d7308b9a660db
I20260709 13:15:01.103891  1003 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:01.104692  6340 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:01.105361   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet ad6ca2a4a8134a10b60d7308b9a660db (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:01.111548  6340 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:01.118640  6433 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:01.123508  1002 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.131295  1599 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:01.130599  1003 raft_consensus.cc:2955] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:01.149477   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dcb3d540c3fc49bf926540f7bd349110 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:01.152361  6462 ts_tablet_manager.cc:1929] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:15:01.158731   752 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 12 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:01.170503  6462 log.cc:1199] T cfd817716cdc4448b3f884eecc94351c P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/cfd817716cdc4448b3f884eecc94351c
I20260709 13:15:01.172402   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet cfd817716cdc4448b3f884eecc94351c (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:01.184051   919 tablet_service.cc:1558] Processing DeleteTablet for tablet dcb3d540c3fc49bf926540f7bd349110 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:42376
I20260709 13:15:01.185207  6464 tablet_replica.cc:333] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:01.186342  6464 raft_consensus.cc:2243] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.187160  6464 raft_consensus.cc:2272] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.191831  6464 ts_tablet_manager.cc:1916] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:01.215494   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:01.216250   922 raft_consensus.cc:3055] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:01.216635   922 raft_consensus.cc:740] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:01.217414   922 consensus_queue.cc:260] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.218485   922 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:01.218521   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:15:01.219172   850 raft_consensus.cc:3060] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:01.226140   996 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:15:01.227695   821 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:15:01.228747  6340 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:01.229477  6340 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:15:01.230360  6340 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.125334   928 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 14.
I20260709 13:15:01.257856  1002 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:01.260610   922 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:01.264051   850 raft_consensus.cc:2468] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:01.268080   967 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:01.268946  6376 raft_consensus.cc:2804] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:01.269876  6376 raft_consensus.cc:697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:01.270869  6376 consensus_queue.cc:237] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:01.336933   752 catalog_manager.cc:5697] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:01.361207   751 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 13 to 14, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:01.395509  6464 ts_tablet_manager.cc:1929] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 13:15:01.398976  6464 log.cc:1199] T dcb3d540c3fc49bf926540f7bd349110 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/dcb3d540c3fc49bf926540f7bd349110
I20260709 13:15:01.414870   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet dcb3d540c3fc49bf926540f7bd349110 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:01.671680  1003 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:01.674659  6433 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:01.698005   928 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:01.699972  6340 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:01.728988   850 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:01.744597  6384 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:01.840438   930 raft_consensus.cc:1275] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:15:01.842880  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 54492.9us
I20260709 13:15:01.844013  6384 consensus_queue.cc:1048] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:15:01.875041   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 74189.5us
W20260709 13:15:01.908479   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55650us
W20260709 13:15:01.997156  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 75536.9us
I20260709 13:15:02.009223 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:02.441931   923 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:15:02.470386   894 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 731bda621b364aaa81d868d0cef5e739. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:02.472743  1003 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:02.477142  6395 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:02.477686   847 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:02.479641  6395 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:02.757421  6431 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:02.830716   847 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:02.820523   996 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:02.852914  1003 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:02.842870   751 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:02.913904   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 72748.1us
I20260709 13:15:03.019460   930 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:03.040117  6496 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:15:03.055828   968 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6533a74f2a524baab75bdfae9b5b1888. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.080943   850 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:03.099269  6503 ts_tablet_manager.cc:933] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:03.108355 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:03.117794  6497 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.119516   930 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.134541  1003 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:15:03.138042   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52708.5us
I20260709 13:15:03.162837  6503 tablet_copy_client.cc:323] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:15:03.153677  6499 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:15:03.165210   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 731bda621b364aaa81d868d0cef5e739 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:15:03.165812   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 731bda621b364aaa81d868d0cef5e739 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-731bda621b364aaa81d868d0cef5e739
I20260709 13:15:03.179898   940 tablet_copy_source_session.cc:215] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:03.183859  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731bda621b364aaa81d868d0cef5e739. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:03.253162   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 55397.8us
I20260709 13:15:03.252236   847 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:15:03.256215   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 171843us
I20260709 13:15:03.287056   850 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 18 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:15:03.345001  6512 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:15:03.344799  6474 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:03.303226   968 consensus_peers.cc:597] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fe55b34a7fd4408b9a1e51ded1fd129. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.316215   930 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:15:03.363880   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 61255.6us
W20260709 13:15:03.432965   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 49768us
I20260709 13:15:03.442917  6474 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:03.465402   752 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 9, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.523758  6516 ts_tablet_manager.cc:933] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:03.557725  6503 tablet_copy_client.cc:806] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:03.560297  6503 tablet_copy_client.cc:670] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:03.577921  6516 tablet_copy_client.cc:323] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:15:03.578493  6446 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.588094   852 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.596251  6503 tablet_copy_client.cc:538] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:03.636796  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 6533a74f2a524baab75bdfae9b5b1888 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:15:03.637482  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 6533a74f2a524baab75bdfae9b5b1888 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-6533a74f2a524baab75bdfae9b5b1888
I20260709 13:15:03.639430   752 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 13, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 18 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.650964   922 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:03.661846   928 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:15:03.674605   894 consensus_peers.cc:597] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35b46c75a1e049dd910275709bd62839. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.674630  1002 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:03.677068  6449 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:03.677968   846 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:03.681006  6449 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:03.652992  1014 tablet_copy_source_session.cc:215] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:03.700738  6491 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.702229   852 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.721673  6516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6533a74f2a524baab75bdfae9b5b1888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:03.740289  1002 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Committing config change with OpId 9.6: config changed from index -1 to 6, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.760293  6503 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:03.770627   750 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 6, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:03.819928  6522 ts_tablet_manager.cc:933] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:03.848433  6522 tablet_copy_client.cc:323] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:15:03.851538  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:15:03.852138  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet 2fe55b34a7fd4408b9a1e51ded1fd129 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-2fe55b34a7fd4408b9a1e51ded1fd129
I20260709 13:15:03.853057  6516 tablet_copy_client.cc:806] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:03.853678  6516 tablet_copy_client.cc:670] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:03.870994  1002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:03.871750  1002 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:03.874958  1014 tablet_copy_source_session.cc:215] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:03.875259  6516 tablet_copy_client.cc:538] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:03.881438  6522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe55b34a7fd4408b9a1e51ded1fd129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:03.888365   852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:15:03.889024   852 raft_consensus.cc:606] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:15:03.890822 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:03.892962 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:03.922420  6516 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:04.000218  6522 tablet_copy_client.cc:806] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.000841  6522 tablet_copy_client.cc:670] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.030563  6522 tablet_copy_client.cc:538] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.050472  6503 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.063040  6522 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:04.078588  6503 tablet_bootstrap.cc:492] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:04.079488  6503 ts_tablet_manager.cc:1403] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.319s	user 0.060s	sys 0.047s
I20260709 13:15:04.082859  6503 raft_consensus.cc:359] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.086567  6516 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.088714  6516 tablet_bootstrap.cc:492] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:04.091267  6516 ts_tablet_manager.cc:1403] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.169s	user 0.028s	sys 0.015s
I20260709 13:15:04.094259  6503 raft_consensus.cc:740] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:04.096655  6503 consensus_queue.cc:260] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.095404  6516 raft_consensus.cc:359] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.103091  6516 raft_consensus.cc:740] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:04.103907  6516 consensus_queue.cc:260] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.105778  6503 ts_tablet_manager.cc:1434] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:15:04.110189   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-731bda621b364aaa81d868d0cef5e739 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:15:04.110714   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-731bda621b364aaa81d868d0cef5e739 on tablet 731bda621b364aaa81d868d0cef5e739 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:04.107805  6516 ts_tablet_manager.cc:1434] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:15:04.114387  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-6533a74f2a524baab75bdfae9b5b1888 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:15:04.114828  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-6533a74f2a524baab75bdfae9b5b1888 on tablet 6533a74f2a524baab75bdfae9b5b1888 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:04.121362  6474 raft_consensus.cc:993] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:04.121945  6474 raft_consensus.cc:1081] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:04.124580   928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42382
I20260709 13:15:04.125147   928 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:04.125528   928 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:04.152637   928 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.174353   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:15:04.175676   852 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:04.194401  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:04.195061  1003 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:04.195343  1003 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:04.196183  1003 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.197410   928 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 19 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:04.198083  1003 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:04.201362   852 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 19.
I20260709 13:15:04.208400  6487 raft_consensus.cc:993] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:04.209241  6487 raft_consensus.cc:1081] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:04.211736  1002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:15:04.212280  1002 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:04.212648  1002 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:04.222002  1002 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:04.222420  1003 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 19.
I20260709 13:15:04.224172   896 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:04.225311  6395 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:04.225481   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:04.225893  6395 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:04.226137   928 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:04.226431   846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:15:04.227062   846 raft_consensus.cc:3055] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:04.228017   846 raft_consensus.cc:740] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:15:04.228814   846 consensus_queue.cc:260] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:04.227026  6395 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.229993   846 raft_consensus.cc:3060] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:04.239624   928 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:04.241062   967 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:04.241881  6474 raft_consensus.cc:2804] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:04.242409  6474 raft_consensus.cc:697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:04.243294  6474 consensus_queue.cc:237] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:04.248703  1002 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:04.254609   846 raft_consensus.cc:2468] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:04.255240   751 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 18 to 19, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 19 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.271554   752 catalog_manager.cc:5697] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 14, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:04.274230  6516 ts_tablet_manager.cc:933] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:04.276412  6522 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.282801  6522 tablet_bootstrap.cc:492] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:04.283569  6522 ts_tablet_manager.cc:1403] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.221s	user 0.054s	sys 0.018s
I20260709 13:15:04.310324  6522 raft_consensus.cc:359] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.311255  6522 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:04.347774  6522 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.349207  6516 tablet_copy_client.cc:323] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:15:04.364324   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 35b46c75a1e049dd910275709bd62839 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:15:04.365422   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 35b46c75a1e049dd910275709bd62839 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-35b46c75a1e049dd910275709bd62839
I20260709 13:15:04.366472  6522 ts_tablet_manager.cc:1434] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.011s
I20260709 13:15:04.369135  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-2fe55b34a7fd4408b9a1e51ded1fd129 received from {username='slave'} at 127.0.0.1:33108
I20260709 13:15:04.369529  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-2fe55b34a7fd4408b9a1e51ded1fd129 on tablet 2fe55b34a7fd4408b9a1e51ded1fd129 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:04.398059   940 tablet_copy_source_session.cc:215] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:04.404739  6516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b46c75a1e049dd910275709bd62839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:04.443504  1596 raft_consensus.cc:1217] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:15:04.472149  6516 tablet_copy_client.cc:806] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.473430  6516 tablet_copy_client.cc:670] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.483673  6516 tablet_copy_client.cc:538] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.510066  6516 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:04.590628  1596 raft_consensus.cc:1217] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 13:15:04.640599  6516 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.641645  6516 tablet_bootstrap.cc:492] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:04.648154  6516 ts_tablet_manager.cc:1403] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.138s	user 0.028s	sys 0.005s
I20260709 13:15:04.651113  6516 raft_consensus.cc:359] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.652009  6516 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:04.652660  6516 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:04.666982  1596 raft_consensus.cc:1217] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 LEARNER]: Deduplicated request from leader. Original: 9.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 13:15:04.679659  6516 ts_tablet_manager.cc:1434] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:15:04.691062   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-35b46c75a1e049dd910275709bd62839 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:15:04.691571   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-35b46c75a1e049dd910275709bd62839 on tablet 35b46c75a1e049dd910275709bd62839 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:04.697950  1003 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:15:04.699542  6395 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:04.758762   930 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:04.763000  6474 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:15:04.794442  1596 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 18 LEARNER]: Advancing to term 19
I20260709 13:15:04.802167   852 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:15:04.804329  6491 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:04.808475   852 raft_consensus.cc:1275] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:04.810228  6474 consensus_queue.cc:1048] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:04.816741  1596 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:15:04.894289 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:04.893081  6395 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:04.978819   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 114525us
W20260709 13:15:04.985582   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 53212.6us
I20260709 13:15:04.975479  6395 raft_consensus.cc:1064] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:05.061674  6395 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:05.068327  6446 raft_consensus.cc:1064] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:05.218568  6446 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:05.231496  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50513.6us
W20260709 13:15:05.259233  6539 log.cc:927] Time spent T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:15:05.293462   846 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:05.306435   930 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:05.313532  1596 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:05.316493   852 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:05.327898  6449 raft_consensus.cc:1064] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:05.329128  6474 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:05.329494  1592 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:05.330878  6449 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 4, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:05.327970  6538 raft_consensus.cc:1064] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:05.335410  6538 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:05.365303   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70003.6us
W20260709 13:15:05.373448   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58671.5us
I20260709 13:15:05.385926  1001 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:05.385495  6474 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:15:05.439368  1003 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:05.442080   852 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:05.442704   847 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:05.436340  6449 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:15:05.444481  6499 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:05.445107  6395 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:05.449564  6538 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:05.452751  1003 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:05.463935  6491 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:15:05.480224   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 95697.5us
W20260709 13:15:05.491575  6552 log.cc:927] Time spent T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:05.481099  6547 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:15:05.527478  6541 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:05.532105  6538 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:15:05.521101   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 95077.4us
I20260709 13:15:05.544082   855 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.596951   750 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 13 to 15, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:15:05.615785   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 121948us
W20260709 13:15:05.611138   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64092.2us
W20260709 13:15:05.641486  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 320829us
I20260709 13:15:05.651296  6542 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Committing config change with OpId 9.7: config changed from index 6 to 7, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.656062  6547 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:05.671705   997 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.670404  6555 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.668258  1003 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.695640   846 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.708309  6540 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.725116   852 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.736446  1592 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.741806   850 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.750860  1599 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:05.756258  1596 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:05.765115   923 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.781399  6395 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:05.775774  6547 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:15:05.866183   997 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.878660  1598 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.904317  1594 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 6 to 7, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.915781  1596 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:05.941005   749 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.040728   752 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.042484   752 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 6 to 7, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.110260   930 consensus_queue.cc:237] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.120487  1596 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:06.121054   846 raft_consensus.cc:1275] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:06.125490  6395 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:06.128933  1003 consensus_queue.cc:237] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.131361  6491 consensus_queue.cc:1048] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:06.136570  6491 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.140648   846 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.144290  1598 raft_consensus.cc:2955] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.151793   850 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:06.153892  6446 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:06.163074  6446 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.164299   749 catalog_manager.cc:5697] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.175145   847 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.182299   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 731bda621b364aaa81d868d0cef5e739 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.204639  1596 raft_consensus.cc:1275] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:06.214318  6556 consensus_queue.cc:1048] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:06.223142   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6533a74f2a524baab75bdfae9b5b1888 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.280025  1596 raft_consensus.cc:2955] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.280766   752 catalog_manager.cc:5697] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 10 to 11, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.289906   994 tablet_service.cc:1558] Processing DeleteTablet for tablet 731bda621b364aaa81d868d0cef5e739 with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51112
I20260709 13:15:06.326629  6575 tablet_replica.cc:333] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:06.336223   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 6533a74f2a524baab75bdfae9b5b1888 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:42376
I20260709 13:15:06.343327  6575 raft_consensus.cc:2243] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.361390  6575 raft_consensus.cc:2272] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.389398  6575 ts_tablet_manager.cc:1916] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.392051  6576 tablet_replica.cc:333] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:06.394049  6576 raft_consensus.cc:2243] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.394989  6576 raft_consensus.cc:2272] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.401960  6576 ts_tablet_manager.cc:1916] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.418851   928 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.421921  6576 ts_tablet_manager.cc:1929] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:15:06.422564  6576 log.cc:1199] T 6533a74f2a524baab75bdfae9b5b1888 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/6533a74f2a524baab75bdfae9b5b1888
I20260709 13:15:06.424665   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 6533a74f2a524baab75bdfae9b5b1888 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:06.430670   930 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.441124  1003 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:15:06.444958  1598 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:06.445538  1596 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:15:06.446483   997 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:06.447742  6540 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:06.449843  6540 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:06.452997  6395 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:06.454730  6395 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:06.484953 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:06.489621  6547 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.490079  6395 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 7 to 8, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.511003   997 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.509503  1003 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.514967  1598 raft_consensus.cc:2955] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.521986  6575 ts_tablet_manager.cc:1929] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
I20260709 13:15:06.526556  6575 log.cc:1199] T 731bda621b364aaa81d868d0cef5e739 P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/731bda621b364aaa81d868d0cef5e739
I20260709 13:15:06.531013   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet 731bda621b364aaa81d868d0cef5e739 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:06.585572   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35b46c75a1e049dd910275709bd62839 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.586333   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.604182   749 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 15 to 16, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 19 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.625211   930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:15:06.625877   930 raft_consensus.cc:606] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:15:06.629465  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:06.630160  1001 raft_consensus.cc:606] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Received request to transfer leadership to TS 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:06.632176 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:06.634176 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:06.646780  1593 raft_consensus.cc:2955] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.680553   752 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 7 to 8, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 9 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:06.695329   843 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fe55b34a7fd4408b9a1e51ded1fd129 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:39024
I20260709 13:15:06.734344   844 tablet_service.cc:1558] Processing DeleteTablet for tablet 35b46c75a1e049dd910275709bd62839 with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:39024
I20260709 13:15:06.750309  6578 tablet_replica.cc:333] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:06.754213  6578 raft_consensus.cc:2243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.758533  6578 raft_consensus.cc:2272] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.763828  6578 ts_tablet_manager.cc:1916] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.798203  6446 raft_consensus.cc:993] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:15:06.798691  6446 raft_consensus.cc:1081] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Signalling peer 64f2cb5b0e174002811ef197f9eec829 to start an election
I20260709 13:15:06.803411  6578 ts_tablet_manager.cc:1929] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.15
I20260709 13:15:06.803660   850 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
 from {username='slave'} at 127.0.0.1:46360
I20260709 13:15:06.804046  6578 log.cc:1199] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/2fe55b34a7fd4408b9a1e51ded1fd129
I20260709 13:15:06.804174   850 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.804616   850 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.806437   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 2fe55b34a7fd4408b9a1e51ded1fd129 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:06.814677  6578 tablet_replica.cc:333] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:06.815308  6547 raft_consensus.cc:993] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:06.818908  6578 raft_consensus.cc:2243] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.822974  6578 raft_consensus.cc:2272] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.826663  6547 raft_consensus.cc:1081] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:06.828847  1003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51144
I20260709 13:15:06.829383  1003 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.829720  1003 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.833913  6578 ts_tablet_manager.cc:1916] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.841427   850 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.844570  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:06.845299  1596 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.844631   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:06.847858   996 raft_consensus.cc:3055] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:06.848150   996 raft_consensus.cc:740] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:06.849160   996 consensus_queue.cc:260] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.850296   996 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.861380  1003 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.868914   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:06.869737   928 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:06.870126   928 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:06.870188  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:06.870790  1593 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.871307   928 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.872617   928 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.873082   850 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:06.875389   996 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:15:06.876914   821 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:15:06.877470  1593 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 17.
I20260709 13:15:06.877840  6554 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:06.869171  1003 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:06.880239   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:06.878479  6554 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Leader. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:15:06.872538  1596 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64f2cb5b0e174002811ef197f9eec829 in term 13.
I20260709 13:15:06.883260  6554 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.885574  6446 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:06.886158  6446 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:06.886931  6446 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:06.879506   928 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 17.
I20260709 13:15:06.894918  6578 ts_tablet_manager.cc:1929] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.7
I20260709 13:15:06.902465  6578 log.cc:1199] T 35b46c75a1e049dd910275709bd62839 P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/35b46c75a1e049dd910275709bd62839
I20260709 13:15:06.904536   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet 35b46c75a1e049dd910275709bd62839 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:06.924998   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 16 to 17, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 17 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:06.924574   750 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: term changed from 12 to 13, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1). New cstate: current_term: 13 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:07.277958  1596 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:07.279740  6560 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:07.330628   928 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:07.334287  6496 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:07.470695  1596 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:07.473439  6487 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:07.511816  1003 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:07.513418  6553 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:07.635510 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:07.859411  1003 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:15:07.900555   968 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d548ef2981894fe6972da35c997f5cff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.912258   928 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:07.918608  6556 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:07.924031   847 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:07.925730  6446 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:07.951668  6446 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:07.957727   847 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:07.985313   749 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:07.995647   930 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:07.986060   850 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:08.064574  1003 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:08.066040   930 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:08.068856  6591 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:15:08.072479   819 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3247a08a45747539334328f4abf2acc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.076459  6591 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:08.125838  6553 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.150409   930 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.162191  1003 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.199198   930 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:08.229364   750 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 7, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:08.330446   894 consensus_peers.cc:597] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d45ee7bf01de4a3ebb2eb56dcd597d5c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.332466   847 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:08.332639  1003 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:08.334075  6540 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:08.335964  6540 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:08.359818  6540 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.370615   996 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.371343   850 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.391439   750 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.410286   850 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:15:08.493086   819 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2840f4bbf21443318b9ad41dc7e3e853. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.495795   996 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:08.497735  6604 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:08.526371  6610 ts_tablet_manager.cc:933] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:08.548448   930 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:08.550094  6603 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:08.560415  6553 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Committing config change with OpId 10.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.567030   928 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Committing config change with OpId 10.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.569492   996 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Committing config change with OpId 10.8: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.629245   749 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 8, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:08.647131  6610 tablet_copy_client.cc:323] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:15:08.649037  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet d548ef2981894fe6972da35c997f5cff from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:15:08.649565  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet d548ef2981894fe6972da35c997f5cff from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-d548ef2981894fe6972da35c997f5cff
I20260709 13:15:08.661384  1014 tablet_copy_source_session.cc:215] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.686628  6610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d548ef2981894fe6972da35c997f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.696756  6614 ts_tablet_manager.cc:933] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:15:08.733491  6614 tablet_copy_client.cc:323] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:15:08.735323   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet c3247a08a45747539334328f4abf2acc from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:15:08.735906   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet c3247a08a45747539334328f4abf2acc from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-c3247a08a45747539334328f4abf2acc
I20260709 13:15:08.736406 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:08.710703  6610 tablet_copy_client.cc:806] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.737383  6610 tablet_copy_client.cc:670] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.741457  6610 tablet_copy_client.cc:538] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.756047   865 tablet_copy_source_session.cc:215] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.760375  6614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3247a08a45747539334328f4abf2acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.782455  6614 tablet_copy_client.cc:806] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.783038  6614 tablet_copy_client.cc:670] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.796330  6610 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:08.811453  6614 tablet_copy_client.cc:538] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.885490  6616 ts_tablet_manager.cc:933] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:08.917896   996 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:08.918545   996 raft_consensus.cc:606] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Received request to transfer leadership to TS 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:08.919971  6614 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:08.925319   850 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a"
dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51502
I20260709 13:15:08.926003   850 raft_consensus.cc:606] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
I20260709 13:15:08.928463 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:08.930055 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:08.986106  6623 ts_tablet_manager.cc:933] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:15:08.991245  6616 tablet_copy_client.cc:323] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
I20260709 13:15:08.999735  6553 raft_consensus.cc:993] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:09.034150  6553 raft_consensus.cc:1081] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:09.036777   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
I20260709 13:15:09.037389   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet d45ee7bf01de4a3ebb2eb56dcd597d5c from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-d45ee7bf01de4a3ebb2eb56dcd597d5c
I20260709 13:15:09.027838  6610 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=1 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:09.039309   996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51128
I20260709 13:15:09.040102   996 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:09.040591   996 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.052892   940 tablet_copy_source_session.cc:215] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:09.061751   996 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:09.093196  6623 tablet_copy_client.cc:323] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:15:09.113547   996 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:15:09.115556   852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:15:09.116475   852 raft_consensus.cc:3055] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:09.116873   852 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64f2cb5b0e174002811ef197f9eec829, State: Running, Role: LEADER
I20260709 13:15:09.118410   852 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:09.120083   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet 2840f4bbf21443318b9ad41dc7e3e853 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:15:09.125917   852 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.076423  6610 tablet_bootstrap.cc:492] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:09.126223   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet 2840f4bbf21443318b9ad41dc7e3e853 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-2840f4bbf21443318b9ad41dc7e3e853
I20260709 13:15:09.124961  6616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:09.137219   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:09.143877   925 raft_consensus.cc:3060] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.137780  6560 raft_consensus.cc:993] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:09.146677   852 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:09.152091   969 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:15:09.162824  6446 raft_consensus.cc:1081] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:09.165032  6446 raft_consensus.cc:2804] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:09.165604  6446 raft_consensus.cc:697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:09.161533  6610 ts_tablet_manager.cc:1403] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.365s	user 0.045s	sys 0.009s
W20260709 13:15:09.183739  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63001.2us
I20260709 13:15:09.204159   925 raft_consensus.cc:2468] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:09.204368  6446 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:09.230978  6610 raft_consensus.cc:359] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:09.243222  6610 raft_consensus.cc:740] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:09.244921  6610 consensus_queue.cc:260] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:09.248479  6610 ts_tablet_manager.cc:1434] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.000s
I20260709 13:15:09.249419   865 tablet_copy_source_session.cc:215] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:15:09.250376   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 99621.1us
I20260709 13:15:09.250633  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-d548ef2981894fe6972da35c997f5cff received from {username='slave'} at 127.0.0.1:33108
I20260709 13:15:09.251153  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-d548ef2981894fe6972da35c997f5cff on tablet d548ef2981894fe6972da35c997f5cff with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:09.263001  6614 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=1 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:09.265861  6614 tablet_bootstrap.cc:492] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:09.266413  6623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2840f4bbf21443318b9ad41dc7e3e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:09.304656   749 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 13 to 14, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:09.315743  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 151306us
I20260709 13:15:09.378014  6616 tablet_copy_client.cc:806] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:09.321040  6614 ts_tablet_manager.cc:1403] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.401s	user 0.027s	sys 0.012s
W20260709 13:15:09.424878   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 265095us
I20260709 13:15:09.415381  6616 tablet_copy_client.cc:670] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
W20260709 13:15:09.482010   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 78415.4us
I20260709 13:15:09.493734  6616 tablet_copy_client.cc:538] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:15:09.503585   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 98646.8us
I20260709 13:15:09.540526   925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42382
I20260709 13:15:09.541090   925 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:09.541462   925 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:09.547380  6614 raft_consensus.cc:359] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:09.548863  6614 raft_consensus.cc:740] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:09.549607  6614 consensus_queue.cc:260] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:09.552405  6614 ts_tablet_manager.cc:1434] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.133s	user 0.003s	sys 0.003s
I20260709 13:15:09.555202   925 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:09.565464  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:09.566026  1001 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:09.566325  1001 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:09.567061  1001 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:09.568481  1001 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:09.580403  1001 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 18.
I20260709 13:15:09.593537  6623 tablet_copy_client.cc:806] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:09.594272  6623 tablet_copy_client.cc:670] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:09.598572  6623 tablet_copy_client.cc:538] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:09.580840   925 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:09.602596  6616 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:09.607154  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:09.607937  1596 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:09.605997   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:09.614002  6491 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:09.627594  6623 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:09.627334  1596 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 18.
I20260709 13:15:09.631505  6491 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:09.632318   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-c3247a08a45747539334328f4abf2acc received from {username='slave'} at 127.0.0.1:43438
I20260709 13:15:09.632928   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-c3247a08a45747539334328f4abf2acc on tablet c3247a08a45747539334328f4abf2acc with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:09.632970  6491 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:09.673321  1596 raft_consensus.cc:1217] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:15:09.703282   850 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:09.704783  6635 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:09.724479   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 17 to 18, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 18 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:09.747221  1596 raft_consensus.cc:1217] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 13:15:09.793649   923 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:09.802145  6635 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:09.943084 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:10.071928  6623 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:10.073040  6623 tablet_bootstrap.cc:492] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:10.073880  6623 ts_tablet_manager.cc:1403] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.447s	user 0.039s	sys 0.024s
I20260709 13:15:10.098963  6616 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:10.100941  6616 tablet_bootstrap.cc:492] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:10.104122  6616 ts_tablet_manager.cc:1403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.502s	user 0.041s	sys 0.017s
I20260709 13:15:10.104570  6623 raft_consensus.cc:359] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:10.111045  6623 raft_consensus.cc:740] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:10.113361  6623 consensus_queue.cc:260] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:10.138154  6632 raft_consensus.cc:1064] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:10.116979  6616 raft_consensus.cc:359] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:10.149194  6616 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:10.149973  6623 ts_tablet_manager.cc:1434] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.001s
I20260709 13:15:10.153209  6616 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:10.155565   865 tablet_copy_service.cc:342] P 64f2cb5b0e174002811ef197f9eec829: Request end of tablet copy session 597687e464174d1ea6043161e037d240-2840f4bbf21443318b9ad41dc7e3e853 received from {username='slave'} at 127.0.0.1:43438
I20260709 13:15:10.156035   865 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-2840f4bbf21443318b9ad41dc7e3e853 on tablet 2840f4bbf21443318b9ad41dc7e3e853 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:10.157155  6616 ts_tablet_manager.cc:1434] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:15:10.159022   997 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:10.162868   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-d45ee7bf01de4a3ebb2eb56dcd597d5c received from {username='slave'} at 127.0.0.1:46362
I20260709 13:15:10.163293   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-d45ee7bf01de4a3ebb2eb56dcd597d5c on tablet d45ee7bf01de4a3ebb2eb56dcd597d5c with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:10.163897  6638 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:10.142940  6632 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:10.178210  6604 raft_consensus.cc:1064] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:10.197460  6604 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:10.217037  1003 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:10.218235  1594 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:10.219638  1593 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:10.221663   923 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:10.226639  6619 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:10.228403   852 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:10.229347  6636 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:10.231686  6628 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:10.233613   923 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:10.245045  6619 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:15:10.246501  6617 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:15:10.250900  6629 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:10.261731  1599 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:10.280458  6540 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:15:10.332015  6654 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:15:10.351890  6636 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.401381  6632 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.407712   850 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.417809  1001 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.425800  1593 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.428249  1598 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.453265   930 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.471520   752 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.500247   749 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 7 to 8, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:10.628557  1598 raft_consensus.cc:1217] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:15:10.645464  1598 raft_consensus.cc:1217] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 LEARNER]: Deduplicated request from leader. Original: 10.7->[10.8-10.8]   Dedup: 10.8->[]
I20260709 13:15:10.670462   927 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.702770   846 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:10.760214  1594 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:10.761976  6553 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:10.762790  1001 consensus_queue.cc:237] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:10.766393   923 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:10.770856   846 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:10.771473  6553 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:10.772591  6671 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:10.775146  1596 raft_consensus.cc:1275] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:10.778784  6632 consensus_queue.cc:1048] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:10.788053  6628 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.795059   850 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.799150  1594 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.797094  6553 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.804972  1596 raft_consensus.cc:2955] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.818444   753 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 8 to 9, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New cstate: current_term: 11 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.831491   923 raft_consensus.cc:2955] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, VOTER 3bb5726887244caca146e176cc6bd070 (127.31.120.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.834169   994 tablet_service.cc:1558] Processing DeleteTablet for tablet c3247a08a45747539334328f4abf2acc with delete_type TABLET_DATA_TOMBSTONED (TS 3bb5726887244caca146e176cc6bd070 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51112
I20260709 13:15:10.839267   752 catalog_manager.cc:5697] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 11 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:10.847226   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3247a08a45747539334328f4abf2acc with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:10.855629   919 tablet_service.cc:1558] Processing DeleteTablet for tablet d548ef2981894fe6972da35c997f5cff with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:42376
I20260709 13:15:10.882977  6675 tablet_replica.cc:333] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:10.902490  6675 raft_consensus.cc:2243] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:10.903681  6675 raft_consensus.cc:2272] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.905380   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d548ef2981894fe6972da35c997f5cff with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:10.927314  6675 ts_tablet_manager.cc:1916] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:10.947927  6674 tablet_replica.cc:333] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:10.949162  6674 raft_consensus.cc:2243] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:10.959417  6675 ts_tablet_manager.cc:1929] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
I20260709 13:15:10.959939  6675 log.cc:1199] T d548ef2981894fe6972da35c997f5cff P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/d548ef2981894fe6972da35c997f5cff
I20260709 13:15:10.961719   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet d548ef2981894fe6972da35c997f5cff (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:10.985160  6674 raft_consensus.cc:2272] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.995934  6674 ts_tablet_manager.cc:1916] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:11.013082  6540 raft_consensus.cc:1064] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:11.030252  6674 ts_tablet_manager.cc:1929] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.8
I20260709 13:15:11.030826  6674 log.cc:1199] T c3247a08a45747539334328f4abf2acc P 3bb5726887244caca146e176cc6bd070: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/wals/c3247a08a45747539334328f4abf2acc
I20260709 13:15:11.035210   740 catalog_manager.cc:5028] TS 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): tablet c3247a08a45747539334328f4abf2acc (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:11.089720  6540 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.108562 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:11.257330   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 43812.9us
W20260709 13:15:11.264457  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 117855us
I20260709 13:15:11.304646   923 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
mode: GRACEFUL
new_leader_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:40956
I20260709 13:15:11.305361   923 raft_consensus.cc:606] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Received request to transfer leadership to TS 3bb5726887244caca146e176cc6bd070
W20260709 13:15:11.331029   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 46063.3us
I20260709 13:15:11.335795   850 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:11.344193  6491 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:11.398947   996 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:11.416356  6540 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:11.419773  6639 raft_consensus.cc:993] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: : Instructing follower 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:11.420810  6618 raft_consensus.cc:1081] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Signalling peer 3bb5726887244caca146e176cc6bd070 to start an election
I20260709 13:15:11.423907  1594 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:11.435564  1001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:11.436671  1001 raft_consensus.cc:606] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:15:11.455307   997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
 from {username='slave'} at 127.0.0.1:51144
I20260709 13:15:11.456022   997 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:11.456478   997 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:11.466508 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:11.468916  6491 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:11.470551   997 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.471716 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:11.490754  6640 raft_consensus.cc:1064] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:11.502748   997 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:11.504892  6640 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.505780  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:11.506433  1596 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:11.517032   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:11.517836   923 raft_consensus.cc:3055] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:11.518217   923 raft_consensus.cc:740] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:11.519182   923 consensus_queue.cc:260] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.530499   923 raft_consensus.cc:3060] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:11.534464  1596 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 20.
I20260709 13:15:11.536052   968 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:11.538673  6662 raft_consensus.cc:2804] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:11.539387  6662 raft_consensus.cc:697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 20 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:11.540587  6662 consensus_queue.cc:237] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.543871   923 raft_consensus.cc:2468] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 20.
W20260709 13:15:11.547910   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 59125.3us
W20260709 13:15:11.564823  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 95007us
I20260709 13:15:11.617589   852 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.610675  6639 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.669886  1003 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:11.671622  6640 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:11.680164   927 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:11.690871   753 catalog_manager.cc:5697] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 19 to 20, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 20 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:11.705374  6640 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:11.736586  1593 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:11.755847  6603 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:11.761574  6679 raft_consensus.cc:993] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:15:11.762101  6662 raft_consensus.cc:1081] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:15:11.771009  1593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:15:11.771664  1593 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:11.772066  1593 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:11.786988   997 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.794868   751 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.798436  1593 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.840939   850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "64f2cb5b0e174002811ef197f9eec829"
I20260709 13:15:11.841645   850 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:11.844077  1599 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.845856  6636 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.851794   850 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:15:11.848439  1003 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.854522  1566 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829; no voters: 
I20260709 13:15:11.856556  6684 raft_consensus.cc:2804] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:11.857165  6684 raft_consensus.cc:697] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:11.858062  6684 consensus_queue.cc:237] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.860383  1593 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 6 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:11.862849   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:11.863492   997 raft_consensus.cc:3055] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:11.863875   997 raft_consensus.cc:740] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:11.868471  1594 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.895875   997 consensus_queue.cc:260] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:11.897087   997 raft_consensus.cc:3060] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:11.909835   997 raft_consensus.cc:2468] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 6.
I20260709 13:15:11.932844   923 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:11.965559   750 catalog_manager.cc:5697] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 5 to 6, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 6 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:12.039705  1593 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:15:12.042330  6662 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:15:12.070981   749 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 8 to 9, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.121079   923 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:12.121359   846 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:12.130012  1599 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:15:12.130012  1594 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:12.132961  6685 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:12.136824   927 raft_consensus.cc:1275] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:15:12.136956  6638 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:12.142406  6671 consensus_queue.cc:1048] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:12.142711  6685 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Committing config change with OpId 10.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.146369   997 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:15:12.145826  6618 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.148118  6638 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:12.147933  1599 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Committing config change with OpId 10.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.154109   997 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.159003  1003 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:15:12.164252  1594 raft_consensus.cc:2955] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.164301  6640 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:15:12.194337   996 raft_consensus.cc:2955] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Committing config change with OpId 10.10: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.213305   751 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.233531   737 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:12.252907   739 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2840f4bbf21443318b9ad41dc7e3e853 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:12.264376  1003 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:12.265806   843 tablet_service.cc:1558] Processing DeleteTablet for tablet d45ee7bf01de4a3ebb2eb56dcd597d5c with delete_type TABLET_DATA_TOMBSTONED (TS 64f2cb5b0e174002811ef197f9eec829 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39024
I20260709 13:15:12.271282   751 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 9 to 10, VOTER 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) evicted. New cstate: current_term: 10 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:12.278641  6684 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:12.298913   919 tablet_service.cc:1558] Processing DeleteTablet for tablet 2840f4bbf21443318b9ad41dc7e3e853 with delete_type TABLET_DATA_TOMBSTONED (TS 91ff541db50a49efa03669dfd777fc8e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:42376
I20260709 13:15:12.329253  6705 tablet_replica.cc:333] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:12.330687   850 raft_consensus.cc:1275] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:12.333325  6676 consensus_queue.cc:1048] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:12.347362  6703 tablet_replica.cc:333] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:12.350800  6703 raft_consensus.cc:2243] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:12.351758  6703 raft_consensus.cc:2272] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:12.356298  6703 ts_tablet_manager.cc:1916] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:12.357980  6705 raft_consensus.cc:2243] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:12.366369  6705 raft_consensus.cc:2272] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:12.379662  6705 ts_tablet_manager.cc:1916] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:12.401651  6705 ts_tablet_manager.cc:1929] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.9
I20260709 13:15:12.402191  6705 log.cc:1199] T 2840f4bbf21443318b9ad41dc7e3e853 P 91ff541db50a49efa03669dfd777fc8e: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/wals/2840f4bbf21443318b9ad41dc7e3e853
I20260709 13:15:12.404155   737 catalog_manager.cc:5028] TS 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): tablet 2840f4bbf21443318b9ad41dc7e3e853 (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:12.411461  6703 ts_tablet_manager.cc:1929] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
I20260709 13:15:12.412017  6703 log.cc:1199] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 64f2cb5b0e174002811ef197f9eec829: Deleting WAL directory at /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/wals/d45ee7bf01de4a3ebb2eb56dcd597d5c
I20260709 13:15:12.413832   739 catalog_manager.cc:5028] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): tablet d45ee7bf01de4a3ebb2eb56dcd597d5c (table test-workload [id=503712c4318e4a7b80efce944ee5995b]) successfully deleted
I20260709 13:15:12.473277 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:13.655370 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:13.902377   997 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
new_leader_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:34972
I20260709 13:15:13.903353   997 raft_consensus.cc:606] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Received request to transfer leadership to TS 597687e464174d1ea6043161e037d240
I20260709 13:15:13.905354 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:15:13.909449 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:13.916996   852 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
W20260709 13:15:13.959090   819 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b9d8e1d04724f1ba621ecf90cc49b82. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:13.999768   923 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:14.001739  6712 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:14.048695   997 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:14.068096  6603 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:14.072224  6713 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.100484   923 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.129526  1002 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:14.159736   996 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.161226   752 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.177670   852 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:15:14.179399   968 consensus_peers.cc:597] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bedf1cefa52d44d884e14c5261c6437a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.179459  6662 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:14.182518   923 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:14.184298  6671 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:15:14.191319   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 50828.4us
I20260709 13:15:14.243314  6679 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.247102   852 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.261183   923 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.349429   923 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:14.371871  6679 raft_consensus.cc:993] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:15:14.372632  6679 raft_consensus.cc:1081] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Signalling peer 597687e464174d1ea6043161e037d240 to start an election
I20260709 13:15:14.375725  1599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9"
dest_uuid: "597687e464174d1ea6043161e037d240"
 from {username='slave'} at 127.0.0.1:37802
I20260709 13:15:14.376734  1599 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.377287  1599 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.376722  6730 ts_tablet_manager.cc:933] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377)
I20260709 13:15:14.409407   852 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:15:14.411667   894 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 597687e464174d1ea6043161e037d240 (127.31.120.4:33689): Couldn't send request to peer 597687e464174d1ea6043161e037d240. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fb0c412ac6a42fe84d4b2ce85773e26. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.413307  1002 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:14.415889  6639 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:14.417793  6639 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:14.438647  1599 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:14.454052   751 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 reported cstate change: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.461479  1599 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:14.463851  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:14.465647  1002 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:14.466079  1002 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:15:14.499740   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 65071.6us
I20260709 13:15:14.501564   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:14.502224   923 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.514039   923 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:14.547164  1002 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:14.577751   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 63432.7us
I20260709 13:15:14.591658  1564 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:14.595748  6726 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:14.598249  6726 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:14.599318  6726 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:14.601585  6639 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.611927  1002 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.618253  6730 tablet_copy_client.cc:323] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.1:43377
I20260709 13:15:14.622489  1002 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:14.622618   996 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.625176   865 tablet_copy_service.cc:140] P 64f2cb5b0e174002811ef197f9eec829: Received BeginTabletCopySession request for tablet 5b9d8e1d04724f1ba621ecf90cc49b82 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:43438)
I20260709 13:15:14.625911   865 tablet_copy_service.cc:161] P 64f2cb5b0e174002811ef197f9eec829: Beginning new tablet copy session on tablet 5b9d8e1d04724f1ba621ecf90cc49b82 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:43438: session id = 597687e464174d1ea6043161e037d240-5b9d8e1d04724f1ba621ecf90cc49b82
I20260709 13:15:14.651072   865 tablet_copy_source_session.cc:215] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:14.638442   852 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
W20260709 13:15:14.672022  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 122226us
I20260709 13:15:14.696972  6730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9d8e1d04724f1ba621ecf90cc49b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.699865   752 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index -1 to 10, NON_VOTER 597687e464174d1ea6043161e037d240 (127.31.120.4) added. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } }
I20260709 13:15:14.771939   749 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 12, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.817989  6746 ts_tablet_manager.cc:933] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:14.899927  6746 tablet_copy_client.cc:323] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.3:44411
I20260709 13:15:14.902963  1014 tablet_copy_service.cc:140] P 3bb5726887244caca146e176cc6bd070: Received BeginTabletCopySession request for tablet bedf1cefa52d44d884e14c5261c6437a from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:33108)
I20260709 13:15:14.903582  1014 tablet_copy_service.cc:161] P 3bb5726887244caca146e176cc6bd070: Beginning new tablet copy session on tablet bedf1cefa52d44d884e14c5261c6437a from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:33108: session id = 597687e464174d1ea6043161e037d240-bedf1cefa52d44d884e14c5261c6437a
I20260709 13:15:14.910689 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:14.921653  1014 tablet_copy_source_session.cc:215] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:14.923195  6730 tablet_copy_client.cc:806] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:14.924052  6730 tablet_copy_client.cc:670] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:14.925700  6746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedf1cefa52d44d884e14c5261c6437a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.939447  6730 tablet_copy_client.cc:538] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:14.983036  6730 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:15.039706  6746 tablet_copy_client.cc:806] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:15.040485  6746 tablet_copy_client.cc:670] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:15.050233  6746 tablet_copy_client.cc:538] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:15.081557   996 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:15.083467  6726 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:15.084260  6748 ts_tablet_manager.cc:933] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:15.326786 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:15:15.334735  6746 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:15.335386  6748 tablet_copy_client.cc:323] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: tablet copy: Beginning tablet copy session from remote peer at address 127.31.120.2:34547
W20260709 13:15:15.406723   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 299121us
W20260709 13:15:15.727695  6750 log.cc:927] Time spent T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:15:15.790782  1566 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:15.842947   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 93223.4us
W20260709 13:15:15.915349  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46453.2us
W20260709 13:15:15.939589  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 148940us
I20260709 13:15:15.956736   940 tablet_copy_service.cc:140] P 91ff541db50a49efa03669dfd777fc8e: Received BeginTabletCopySession request for tablet 2fb0c412ac6a42fe84d4b2ce85773e26 from peer 597687e464174d1ea6043161e037d240 ({username='slave'} at 127.0.0.1:46362)
W20260709 13:15:16.052075  6730 log.cc:927] Time spent T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.073s	user 0.004s	sys 0.000s
I20260709 13:15:16.092687   928 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:16.108958   940 tablet_copy_service.cc:161] P 91ff541db50a49efa03669dfd777fc8e: Beginning new tablet copy session on tablet 2fb0c412ac6a42fe84d4b2ce85773e26 from peer 597687e464174d1ea6043161e037d240 at {username='slave'} at 127.0.0.1:46362: session id = 597687e464174d1ea6043161e037d240-2fb0c412ac6a42fe84d4b2ce85773e26
I20260709 13:15:16.155926  6771 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:16.154548  6782 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.171530  6782 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.168344 32224 tablet_copy_service.cc:362] P 64f2cb5b0e174002811ef197f9eec829: Destroying tablet copy session 597687e464174d1ea6043161e037d240-5b9d8e1d04724f1ba621ecf90cc49b82 due to service shutdown
I20260709 13:15:16.182422 32224 tablet_copy_service.cc:434] P 64f2cb5b0e174002811ef197f9eec829: ending tablet copy session 597687e464174d1ea6043161e037d240-5b9d8e1d04724f1ba621ecf90cc49b82 on tablet 5b9d8e1d04724f1ba621ecf90cc49b82 with peer 597687e464174d1ea6043161e037d240
W20260709 13:15:16.210815  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.216487  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.224191  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.224829  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.225380  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.225862  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.226425  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.229537  6782 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:16.231283   940 tablet_copy_source_session.cc:215] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:16.233973  6780 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.234321  6776 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:15:16.235424   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 84347us
I20260709 13:15:16.235862  6715 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.243993  6781 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:15:16.244967   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 85465us
I20260709 13:15:16.244963  6712 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.265722  6715 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.284507  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.294965   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.298359  6748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb0c412ac6a42fe84d4b2ce85773e26. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:16.307849   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.256021  6776 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:16.306165  6767 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.310529  6767 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.311777  6776 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:16.311830   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.312800  6767 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.314090  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.263711  6781 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.320539  6781 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:16.322234   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.328052   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.329866   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:16.333452   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.334156   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.334861   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.335407   969 consensus_peers.cc:597] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.336364  6781 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.337224  6730 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:16.337019  6781 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:16.252339  6780 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.338316  6730 tablet_bootstrap.cc:492] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
W20260709 13:15:16.338224  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 53892.4us
I20260709 13:15:16.339244  6730 ts_tablet_manager.cc:1403] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 1.357s	user 0.055s	sys 0.011s
W20260709 13:15:16.335951   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.280462  6769 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.277056  6715 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.339361  6780 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:16.396958   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 61060.5us
I20260709 13:15:16.367796  6776 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.400813  6776 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.403029  6776 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.391155  6766 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.403954  6781 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.280121  6712 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.403967  6766 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.405915  6712 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.406548  6769 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.406859   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.408986  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.352823   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.409380   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.409574  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.344880  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.410386  6769 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.410951  6769 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:16.344194   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.411520   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:16.411890   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 67754.6us
W20260709 13:15:16.412535   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.413101   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.413412  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:16.413722   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.414297   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.414386   818 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.412563   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.353295   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:16.398079   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.416709   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.417270   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.417761   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.418246   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.418835   969 leader_election.cc:336] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:16.419198   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.413113  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.407263  6766 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.423036  6766 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:16.423753  6782 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:16.350876  6767 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.427817  6767 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:16.414840   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.431077   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.435384   821 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:16.431064   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.437284  6766 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:16.438659   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.433795  6769 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:15:16.453415   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.462386   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.453192  6781 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.466419  6781 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.468667  6781 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.469237   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.448633  6712 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:16.475666  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.436066  6730 raft_consensus.cc:359] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:16.483860   821 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.487807  6767 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.477903  6730 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:16.473790  6769 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.517704   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.522276  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.507409  6769 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.524354  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95c3b97a883a4aeb97bdcc43b82a2b95" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.543150  6762 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:16.515767  6730 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:16.552273  6776 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.578778  6776 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.581409  6776 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.583895  6744 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.584710  6744 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:16.590862   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 74706.4us
I20260709 13:15:16.589588  6712 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.594748  6713 raft_consensus.cc:493] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.595163  6781 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.595340  6713 raft_consensus.cc:515] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.595845  6783 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.595749  6781 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.596397  6783 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.597519  6713 leader_election.cc:290] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.597786  6781 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.598040  6713 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.598469  6783 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.598587  6713 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.600047  6744 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.604791  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.605926  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6533a74f2a524baab75bdfae9b5b1888" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.607231  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.608117  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.608574  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.609978  6715 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.609933  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.610603  6715 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.611683  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.612843  6715 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.613060  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516d927af6bf46aa947c0919f1dfc830" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.616623  6783 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.617179  6783 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.619294  6783 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.614763  6767 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.619885  6769 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:16.636415  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.621366  6730 ts_tablet_manager.cc:1434] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.225s	user 0.008s	sys 0.000s
I20260709 13:15:16.627734  6776 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.628177  6715 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.629899  6713 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.632586  6762 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.637034   821 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.594141  6712 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.640074  6767 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.663112   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.665025  6712 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.642993   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.655437  6769 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.669836   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.650417   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.670588  6713 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:16.650384  6776 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:16.671381  6769 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.672636  6776 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.652287   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.649446  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed094558dad4fb2b45bc747bbc6adba" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.654793   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.661018  6762 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:16.685537   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48575.6us
I20260709 13:15:16.687492   821 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.641731  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.669903   819 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.652683  6715 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.679193  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.663323  6767 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:16.713982  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.723244  6783 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.723925  6744 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:16.726343  6746 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:16.726838   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.727847  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:16.730983   819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 61149.7us
I20260709 13:15:16.737221  6783 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.739487   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.742728   819 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.745393  6783 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.745986  6744 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:16.751060  6746 tablet_bootstrap.cc:492] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:16.751938  6746 ts_tablet_manager.cc:1403] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 1.418s	user 0.043s	sys 0.019s
I20260709 13:15:16.753091   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:16.754616   819 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.757893  6713 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:16.740303   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 52920.3us
I20260709 13:15:16.760167   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.760397  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.761659  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.738539  6712 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:16.765844  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:16.765990  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.774621  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:16.752976  6776 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:16.756006  6783 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:16.756112  6769 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:16.786048   821 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:15:16.819576  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 183137us
I20260709 13:15:16.777560  6713 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:16.784272  6712 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:16.792820  6780 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:16.826965  6712 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:16.860559   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.853299  6715 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:16.882067   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.883714   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.884145   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.884799   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.885301   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.886338   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:16.833148  6713 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:16.952970   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 163796us
I20260709 13:15:16.883327  6746 raft_consensus.cc:359] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:16.954994   821 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.962010   821 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:16.958379  6746 raft_consensus.cc:740] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:16.963622  6715 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:16.973562  6793 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:17.023141  6762 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:17.054689   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 58097us
W20260709 13:15:17.135497   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 79724.5us
I20260709 13:15:17.141691  6712 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:17.143401  6713 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:17.057428  6746 consensus_queue.cc:260] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:17.146068  6746 ts_tablet_manager.cc:1434] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.394s	user 0.008s	sys 0.000s
I20260709 13:15:17.155356   818 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.165534  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:17.167400   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:17.169219  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:17.174767   818 leader_election.cc:304] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.176083   818 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.177224   818 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.177964  6769 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:17.178313   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.178709  6769 raft_consensus.cc:2749] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:17.179507  6769 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:17.179774  6781 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:17.183818  6812 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:17.207437   818 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.214737  6812 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:17.216176   818 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.217566  6812 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:17.219522   818 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.221081   818 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.221903  6811 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:17.225785   821 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.228020  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:17.232259  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:17.222342  6812 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:17.245018  6748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:37790 (request call id 1258) took 2219 ms. Trace:
I20260709 13:15:17.245551  6748 rpcz_store.cc:276] 0709 13:15:15.025513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.025664 (+   151us) service_pool.cc:224] Handling call
0709 13:15:15.084691 (+ 59027us) ts_tablet_manager.cc:934] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Initiating tablet copy from peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
0709 13:15:16.283878 (+1199187us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:15:16.454776 (+170898us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:16.460286 (+  5510us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:15:16.764441 (+304155us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:15:16.785022 (+ 20581us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:17.244864 (+459842us) ts_tablet_manager.cc:989] Replica registered.
0709 13:15:17.244927 (+    63us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":228147,"tablet-copy.queue_time_us":57981,"thread_start_us":58817,"threads_started":2}
I20260709 13:15:17.247534  6748 tablet_copy_client.cc:806] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:17.265990  6748 tablet_copy_client.cc:670] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: tablet copy: Starting download of 1 WAL segments...
W20260709 13:15:17.279070   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 53351.9us
I20260709 13:15:17.295312   818 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.304672   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:17.305006   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:17.317509  1014 tablet_copy_service.cc:342] P 3bb5726887244caca146e176cc6bd070: Request end of tablet copy session 597687e464174d1ea6043161e037d240-bedf1cefa52d44d884e14c5261c6437a received from {username='slave'} at 127.0.0.1:33108
I20260709 13:15:17.318048  1014 tablet_copy_service.cc:434] P 3bb5726887244caca146e176cc6bd070: ending tablet copy session 597687e464174d1ea6043161e037d240-bedf1cefa52d44d884e14c5261c6437a on tablet bedf1cefa52d44d884e14c5261c6437a with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:17.321626   818 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.300900  6776 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:17.327982  6811 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:17.339329  6748 tablet_copy_client.cc:538] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:17.395732  6796 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:17.357542  6756 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:17.431886  6798 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:17.433194  1594 raft_consensus.cc:1217] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.9->[14.10-14.10]   Dedup: 14.10->[]
I20260709 13:15:17.439103  6748 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Bootstrap starting.
I20260709 13:15:17.440958  6756 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:17.432502  6798 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:17.411320  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:17.475651  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 64338.2us
W20260709 13:15:17.465476   818 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 59273.5us
I20260709 13:15:17.478241  1592 raft_consensus.cc:1217] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.9->[14.10-14.10]   Dedup: 14.10->[]
I20260709 13:15:17.482664   819 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:17.497673  6814 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:17.513561   819 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:17.514400  6811 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:17.515218  6756 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:17.535782  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:17.588500  1566 leader_election.cc:336] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:17.593125  1566 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:17.608776  6772 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:17.609335  6772 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:17.607776  6771 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:17.614019  6772 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:17.621498   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:17.631254  6798 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:17.660447  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68446.1us
I20260709 13:15:17.829372  6748 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:17.832732  6748 tablet_bootstrap.cc:492] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Bootstrap complete.
I20260709 13:15:17.833710  6748 ts_tablet_manager.cc:1403] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Time spent bootstrapping tablet: real 0.395s	user 0.034s	sys 0.016s
I20260709 13:15:17.837059  6748 raft_consensus.cc:359] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:17.838106  6748 raft_consensus.cc:740] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Initialized, Role: LEARNER
I20260709 13:15:17.838908  6748 consensus_queue.cc:260] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:17.841918  6748 ts_tablet_manager.cc:1434] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:17.846422 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:15:17.882366   821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 248188us
I20260709 13:15:17.910386   940 tablet_copy_service.cc:342] P 91ff541db50a49efa03669dfd777fc8e: Request end of tablet copy session 597687e464174d1ea6043161e037d240-2fb0c412ac6a42fe84d4b2ce85773e26 received from {username='slave'} at 127.0.0.1:46362
I20260709 13:15:17.910969   940 tablet_copy_service.cc:434] P 91ff541db50a49efa03669dfd777fc8e: ending tablet copy session 597687e464174d1ea6043161e037d240-2fb0c412ac6a42fe84d4b2ce85773e26 on tablet 2fb0c412ac6a42fe84d4b2ce85773e26 with peer 597687e464174d1ea6043161e037d240
I20260709 13:15:17.976567 32224 tablet_replica.cc:333] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:17.977629 32224 raft_consensus.cc:2243] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:17.978487 32224 raft_consensus.cc:2272] T 6533a74f2a524baab75bdfae9b5b1888 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:17.985472 32224 tablet_replica.cc:333] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:17.997462 32224 raft_consensus.cc:2243] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.017930 32224 raft_consensus.cc:2272] T dcb3d540c3fc49bf926540f7bd349110 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.085752  6817 raft_consensus.cc:1064] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:18.105741  6817 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.112597 32224 tablet_replica.cc:333] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.113657 32224 raft_consensus.cc:2243] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.115888 32224 raft_consensus.cc:2272] T 56cfb3f7cad745918e1bc1faf5cc697c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.127756 32224 tablet_replica.cc:333] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.128849 32224 raft_consensus.cc:2243] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.129462 32224 raft_consensus.cc:2272] T 1b4d7e87915947148833db3fe84cf623 P 64f2cb5b0e174002811ef197f9eec829 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.174714 32224 tablet_replica.cc:333] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.180732 32224 raft_consensus.cc:2243] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.181481 32224 raft_consensus.cc:2272] T 516d927af6bf46aa947c0919f1dfc830 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.185600  6823 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.186319  6823 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:18.190079   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.209841  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.232877 32224 tablet_replica.cc:333] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.233907 32224 raft_consensus.cc:2243] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.241022   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.241715 32224 raft_consensus.cc:2272] T e0cca622722b48a7ba611a3b77376f80 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.242553   821 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.244179  6715 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:18.235874  6823 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.256476  1592 raft_consensus.cc:1217] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 13:15:18.283896   930 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:18.287716  6818 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:15:18.302793   969 consensus_peers.cc:597] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:18.306825  1593 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:18.307268  6767 raft_consensus.cc:1064] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:18.309868  6767 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:18.346354  1566 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:18.347258  6776 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.350199  6776 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:18.353458   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.353631  6776 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.354188  1566 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:18.355119   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.356822   818 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.358036  6776 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:18.358808  6801 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:18.363596  6765 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:18.397490 32224 tablet_replica.cc:333] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.399080  6776 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:18.399461 32224 raft_consensus.cc:2243] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.399654  6776 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.400692 32224 raft_consensus.cc:2272] T 2840f4bbf21443318b9ad41dc7e3e853 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.401925  6776 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:18.396857  1566 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:18.409613  1003 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:18.409933  1566 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.415831  6819 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:18.419591  6776 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:18.425828   930 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:18.427439  6776 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:18.457509 32224 tablet_replica.cc:333] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.461987  1593 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.458488   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.470353  6767 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:18.458837  6797 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.470974  6767 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:18.474321   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.463021 32224 raft_consensus.cc:2243] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.476872  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.477188  1599 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 LEARNER]: Refusing update from remote peer 64f2cb5b0e174002811ef197f9eec829: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:15:18.478276 32224 raft_consensus.cc:2272] T 24f44707b1d3466e91403f063187a9b2 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.478960  6767 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.480511  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.484019   821 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.486562 32224 tablet_replica.cc:333] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.484988  6767 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:18.488286 32224 raft_consensus.cc:2243] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.489310 32224 raft_consensus.cc:2272] T 2ec15122641447a3bd48017eb990578d P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.496287 32224 tablet_replica.cc:333] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.501987   923 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.504026  6776 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.506206   819 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:18.507292  6781 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:18.509057  6781 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:15:18.509512   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.510543   929 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.533056  6781 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.533685  6781 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.534550   996 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.536780   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.539484  6796 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.540032  6796 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:18.541990  6796 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.543944   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.542484 32224 raft_consensus.cc:2243] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.546074  6814 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:18.546617  6814 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.548756  6814 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:18.548768   752 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "64f2cb5b0e174002811ef197f9eec829" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.536818  6781 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:18.552086  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.552165  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31088ed18ff54bcea05d848f8c67e437" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.555193   821 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.555492 32224 raft_consensus.cc:2272] T f379c9f3a933409ca08bceb325f43132 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.558882 32224 tablet_replica.cc:333] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.560022 32224 raft_consensus.cc:2243] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.560882 32224 raft_consensus.cc:2272] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.562321  6781 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:18.553608   750 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.580462  6776 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:18.580997  6776 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.583165  6776 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:18.584868 32224 tablet_replica.cc:333] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.585732 32224 raft_consensus.cc:2243] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.586340 32224 raft_consensus.cc:2272] T 248dca75dd144af2a2106e703fe92de4 P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.588281   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.588516  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.598347  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.600124  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.600518 32224 tablet_replica.cc:333] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.601372 32224 raft_consensus.cc:2243] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.601950 32224 raft_consensus.cc:2272] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.604839 32224 tablet_replica.cc:333] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.605466   819 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.606283  6767 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:18.607540  1596 raft_consensus.cc:2955] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:18.616248  6767 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:18.616946  6767 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.620033   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.619319  6767 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:18.621805 32224 raft_consensus.cc:2243] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.622952 32224 raft_consensus.cc:2272] T 633fa6f0c6434c49aeb87667c9d97141 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.625893 32224 tablet_replica.cc:333] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.626673 32224 raft_consensus.cc:2243] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.627193 32224 raft_consensus.cc:2272] T c43ab43025004edcacdb3d8a30106319 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.628517   819 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:18.633026  6767 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:18.633814   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.634686 32224 tablet_replica.cc:333] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.635565 32224 raft_consensus.cc:2243] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.636168 32224 raft_consensus.cc:2272] T cdfea09a06ff4100a0b80619f35f7008 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.636694   819 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
W20260709 13:15:18.637410   819 raft_consensus.cc:2686] T 53d4a96b7f094cf691b31dbe6fb015bd P 64f2cb5b0e174002811ef197f9eec829: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:15:18.639353  6767 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:18.640352  6767 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } }
I20260709 13:15:18.643471   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:18.643674   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.643778  6767 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.645258   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.646704   821 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.647270 32224 tablet_replica.cc:333] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.647859  6767 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:18.648463  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.653973 32224 raft_consensus.cc:2243] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.654723 32224 raft_consensus.cc:2272] T 91555b4ec53349c287c5221b9030c23a P 64f2cb5b0e174002811ef197f9eec829 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.659945 32224 tablet_replica.cc:333] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.660888 32224 raft_consensus.cc:2243] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.661568 32224 raft_consensus.cc:2272] T 2b38ec4f420f4de9883306887a006c24 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.668151  6710 raft_consensus.cc:1064] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: attempting to promote NON_VOTER 597687e464174d1ea6043161e037d240 to VOTER
I20260709 13:15:18.668797 32224 tablet_replica.cc:333] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.669723 32224 raft_consensus.cc:2243] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.670621 32224 raft_consensus.cc:2272] T d548ef2981894fe6972da35c997f5cff P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.670924  6710 consensus_queue.cc:237] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.680200   819 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:18.681402  6767 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:18.681367 32224 tablet_replica.cc:333] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.683009 32224 raft_consensus.cc:2243] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.683579 32224 raft_consensus.cc:2272] T 5ed094558dad4fb2b45bc747bbc6adba P 64f2cb5b0e174002811ef197f9eec829 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.686316 32224 tablet_replica.cc:333] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.691318 32224 raft_consensus.cc:2243] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.692027 32224 raft_consensus.cc:2272] T 31088ed18ff54bcea05d848f8c67e437 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.692629  6776 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.699750   996 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:18.700183  1599 raft_consensus.cc:1275] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 LEARNER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:18.699934  6776 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.702179  6776 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:18.702404  6710 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:18.703835   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:18.704638   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:18.704880  6710 consensus_queue.cc:1048] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:18.708369  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:18.709815 32224 tablet_replica.cc:333] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.710919 32224 raft_consensus.cc:2243] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.715478 32224 raft_consensus.cc:2272] T 90a3708334cc4bd59c73d494d2747d54 P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.718704 32224 tablet_replica.cc:333] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.721359   819 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.723150  6767 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:18.724411 32224 raft_consensus.cc:2243] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.729135 32224 raft_consensus.cc:2272] T 6444551c84374ec98f4becf8b07446ba P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.739905 32224 tablet_replica.cc:333] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.742136 32224 raft_consensus.cc:2243] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.736167  6819 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:18.773407  6819 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.776407  6819 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.779132  6767 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:18.780705 32224 raft_consensus.cc:2272] T a4018887e34a40108555fd3cbaa2c975 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.780375  1002 consensus_queue.cc:237] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.780772  6767 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:18.784019   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.783680 32224 tablet_replica.cc:333] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.786527 32224 raft_consensus.cc:2243] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:18.796638  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.793615  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:18.784682  6767 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.820484 32224 raft_consensus.cc:2272] T ad6ca2a4a8134a10b60d7308b9a660db P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:18.834106  1566 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:18.834671  1566 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:18.834357 32224 tablet_replica.cc:333] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.848412 32224 raft_consensus.cc:2243] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.870496  6851 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:18.878615 32224 raft_consensus.cc:2272] T 7cf198f3795b43ff951a11167bc21580 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:18.892913  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58850.8us
I20260709 13:15:18.932660  1593 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:15:18.937995   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 70754.7us
I20260709 13:15:18.939742   821 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.942385  6776 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:18.932422  6767 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.950174  6767 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:18.952262  6767 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:18.936446  6806 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:15:18.957928   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.960714   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.962744   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:18.963255   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.963663  6776 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:18.964210  6776 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:18.966230  6776 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:18.967803   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.968894   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:18.969587   818 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:18.961501 32224 tablet_replica.cc:333] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:18.973968 32224 raft_consensus.cc:2243] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:18.980600 32224 raft_consensus.cc:2272] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:18.988432   929 raft_consensus.cc:1275] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:15:18.984539  6767 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:19.011018  6844 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.013696  1596 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.017032 32224 tablet_replica.cc:333] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.018100 32224 raft_consensus.cc:2243] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:19.023833   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.026191   740 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bedf1cefa52d44d884e14c5261c6437a with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:19.018719 32224 raft_consensus.cc:2272] T 6874f81648724ea980572af4b6fa79d2 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:19.035109  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 99118us
I20260709 13:15:19.040421   751 catalog_manager.cc:5697] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 reported cstate change: config changed from index 11 to 12, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New cstate: current_term: 14 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.044697  6847 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.052987  6806 consensus_queue.cc:1048] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:15:19.070986   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.072372   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.079741  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 88926.1us
W20260709 13:15:19.112520  1025 debug-util.cc:398] Leaking SignalData structure 0x7b0800302540 after lost signal to thread 32227
I20260709 13:15:19.136215  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:19.141242  6776 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:19.141840  6776 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:19.144783  1025 debug-util.cc:398] Leaking SignalData structure 0x7b08003011c0 after lost signal to thread 787
I20260709 13:15:19.145025  6776 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:19.145545  1025 debug-util.cc:398] Leaking SignalData structure 0x7b080030b0e0 after lost signal to thread 879
W20260709 13:15:19.146062  1025 debug-util.cc:398] Leaking SignalData structure 0x7b08002752e0 after lost signal to thread 880
W20260709 13:15:19.145889   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56504.7us
W20260709 13:15:19.147208  1025 debug-util.cc:398] Leaking SignalData structure 0x7b080017d700 after lost signal to thread 954
W20260709 13:15:19.148438  1025 debug-util.cc:398] Leaking SignalData structure 0x7b08001f3900 after lost signal to thread 1028
I20260709 13:15:19.154178   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:19.154021  1599 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.156975   819 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:19.159199  6806 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.160301   929 raft_consensus.cc:2955] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
W20260709 13:15:19.166342  1025 debug-util.cc:398] Leaking SignalData structure 0x7b0800373b20 after lost signal to thread 1625
W20260709 13:15:19.170572   819 raft_consensus.cc:2686] T 731bda621b364aaa81d868d0cef5e739 P 64f2cb5b0e174002811ef197f9eec829: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:15:19.172071 32224 tablet_replica.cc:333] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.172942 32224 raft_consensus.cc:2243] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.174068 32224 raft_consensus.cc:2272] T 6d9b9f004a6f4293a658cca1bb92e319 P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.174921  6806 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:19.176098   739 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 101075us
I20260709 13:15:19.177047  6806 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.181757  6853 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.183406  6853 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:19.186973   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:19.185019  1003 raft_consensus.cc:2955] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.189518  6853 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:19.197001 32224 tablet_replica.cc:333] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.197947 32224 raft_consensus.cc:2243] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.199097 32224 raft_consensus.cc:2272] T 4997f2c7b33048a5b010a4e0c7866fa2 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.200261   751 catalog_manager.cc:5697] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 reported cstate change: config changed from index 10 to 11, 597687e464174d1ea6043161e037d240 (127.31.120.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:19.202099 32224 tablet_replica.cc:333] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
W20260709 13:15:19.211969   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.212646  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:19.213347  1596 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:19.214475   968 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:15:19.214861   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:15:19.215353  6844 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:19.215888  6844 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.216359  6844 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:19.230592  6844 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.203014 32224 raft_consensus.cc:2243] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.238624  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:19.240263 32224 raft_consensus.cc:2272] T e65c03394bf04e2fb03562b27d85f5b3 P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:19.242259   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.243338 32224 tablet_replica.cc:333] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.244256 32224 raft_consensus.cc:2243] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.245404 32224 raft_consensus.cc:2272] T 85bbb1b886474a538a2f1a32dd91b560 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.252624   819 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:19.254118  6781 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:19.256603  6844 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.258298  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:19.260505  1599 raft_consensus.cc:3060] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:19.270972   895 leader_election.cc:336] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.271499   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:19.272332  6853 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:19.273078 32224 tablet_replica.cc:333] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
W20260709 13:15:19.273511   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.275573   969 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.274010 32224 raft_consensus.cc:2243] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.284147 32224 raft_consensus.cc:2272] T ef39166ba76a41d18b3770b743ea3488 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.287268 32224 tablet_replica.cc:333] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.288133 32224 raft_consensus.cc:2243] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.289507 32224 raft_consensus.cc:2272] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.294803  1599 raft_consensus.cc:2468] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:19.296631   968 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:19.299914  6806 raft_consensus.cc:2804] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:19.303475 32224 tablet_replica.cc:333] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.303694  6806 raft_consensus.cc:697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:19.304378 32224 raft_consensus.cc:2243] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.305009 32224 raft_consensus.cc:2272] T 9155790402004e4f930983841dfd8c64 P 64f2cb5b0e174002811ef197f9eec829 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.304625  6806 consensus_queue.cc:237] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.307793 32224 tablet_replica.cc:333] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.308708 32224 raft_consensus.cc:2243] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.309469 32224 raft_consensus.cc:2272] T c6e1447d2c0c4822aac226064c087c42 P 64f2cb5b0e174002811ef197f9eec829 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:19.323081   969 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.323302 32224 tablet_replica.cc:333] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.324244 32224 raft_consensus.cc:2243] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.324937 32224 raft_consensus.cc:2272] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 64f2cb5b0e174002811ef197f9eec829 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:19.327193   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.337894  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.338407 32224 tablet_replica.cc:333] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.339252 32224 raft_consensus.cc:2243] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.340350 32224 raft_consensus.cc:2272] T c3247a08a45747539334328f4abf2acc P 64f2cb5b0e174002811ef197f9eec829 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.343140 32224 tablet_replica.cc:333] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.343959 32224 raft_consensus.cc:2243] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.345098 32224 raft_consensus.cc:2272] T 701b27559f8e47f9b63e5d220d3d5a58 P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.347077   751 catalog_manager.cc:5697] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.357640 32224 tablet_replica.cc:333] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.370321  6781 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:19.358595 32224 raft_consensus.cc:2243] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.370965  6781 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.371878 32224 raft_consensus.cc:2272] T bdca38d5137a4409b2e61eaf2e13fd7b P 64f2cb5b0e174002811ef197f9eec829 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.373270  6781 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.375008 32224 tablet_replica.cc:333] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.375026   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:19.376026 32224 raft_consensus.cc:2243] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.376622  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "64f2cb5b0e174002811ef197f9eec829" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:19.378605  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.384451   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.396042   819 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64f2cb5b0e174002811ef197f9eec829; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:19.397320 32224 raft_consensus.cc:2272] T 7de58b0d5ad94d869fac3b0491143543 P 64f2cb5b0e174002811ef197f9eec829 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.398419  6781 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:19.400846 32224 tablet_replica.cc:333] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
W20260709 13:15:19.402129   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.410627 32224 raft_consensus.cc:2243] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.436287 32224 raft_consensus.cc:2272] T 6cfaab49f56546a49978c6055f57640f P 64f2cb5b0e174002811ef197f9eec829 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.442523 32224 tablet_replica.cc:333] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.449353 32224 raft_consensus.cc:2243] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.459367 32224 raft_consensus.cc:2272] T 01fc410be7f74515adce264a85731b90 P 64f2cb5b0e174002811ef197f9eec829 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.470299 32224 tablet_replica.cc:333] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.478967 32224 raft_consensus.cc:2243] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.480165 32224 raft_consensus.cc:2272] T cfd817716cdc4448b3f884eecc94351c P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:19.490177   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.493327 32224 tablet_replica.cc:333] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829: stopping tablet replica
I20260709 13:15:19.494159 32224 raft_consensus.cc:2243] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.494858 32224 raft_consensus.cc:2272] T bedf1cefa52d44d884e14c5261c6437a P 64f2cb5b0e174002811ef197f9eec829 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.527880  6806 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.528477  6806 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.531929  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:19.541814   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.551973  6806 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.582893  6856 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.583639  6856 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.586830  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:19.595191   895 leader_election.cc:336] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.595662   895 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:19.596400  6853 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:19.599743  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 56894us
W20260709 13:15:19.603665  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.605217  6846 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.610044  6844 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.610682  6844 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.612906  6844 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.615022  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:19.608894  6846 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:19.619357   969 leader_election.cc:336] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.619994   969 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:19.621291  6846 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:19.621476   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:19.622114   929 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:19.622218  6844 raft_consensus.cc:2749] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:19.623257   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:19.624142  6844 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:19.624557  6844 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.624943  6844 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:19.631019  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.646467   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.651705  6856 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:19.668792  6844 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:19.669994   969 leader_election.cc:336] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.670480   969 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:19.671551   969 leader_election.cc:336] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.674347  6860 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:19.680162   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.688619   969 leader_election.cc:336] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.689303  6844 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:19.690434   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:19.691133   923 raft_consensus.cc:3060] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:19.702170   923 raft_consensus.cc:2468] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:15:19.703454   967 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:19.711496  6860 raft_consensus.cc:2804] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:19.713212  6860 raft_consensus.cc:697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:19.714114  6860 consensus_queue.cc:237] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:19.744501  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.757189  6852 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.757934  6852 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.760188  6852 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:19.762367   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:19.763032   996 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:15:19.765883  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:19.767572  6857 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:19.768010  6857 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.768404  6857 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:19.768203   751 catalog_manager.cc:5697] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 15 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.770236  6801 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.770744  6801 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.773002  6801 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:19.774334   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:19.775002   923 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:19.776227  1564 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:19.786325   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:19.799173  1566 leader_election.cc:336] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:19.800166  1566 leader_election.cc:336] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.801417  6801 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:19.801887  6801 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.802258  6801 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:19.807528  6862 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.808157  6862 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:19.822791  1566 leader_election.cc:336] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:19.824540  6862 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:19.825161   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:19.825886   996 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:15:19.826911  1567 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:19.828154  6852 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:19.828567  6852 raft_consensus.cc:493] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:19.828948  6852 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:19.843151  6857 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:19.861326  1566 leader_election.cc:336] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:19.867017   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.870597   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:19.871313   996 raft_consensus.cc:3060] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:19.876667  1599 raft_consensus.cc:1275] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:19.882542  6844 consensus_queue.cc:1048] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:15:19.892222  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:19.920142  6857 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:19.856880  6801 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:19.963302  6856 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:15:19.949249   969 connection.cc:769] client connection to 127.31.120.1:43377 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:15:19.964073  6856 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.043555   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75836.7us
I20260709 13:15:19.930406   996 raft_consensus.cc:2468] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:15:20.051290  6856 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.102140  6801 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:20.106470  6730 tablet_copy_client.cc:1131] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.126538   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.127185   929 raft_consensus.cc:3060] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:20.118068  6872 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.135316  6844 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.148866  6852 raft_consensus.cc:515] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.181313   929 raft_consensus.cc:2468] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 13.
I20260709 13:15:20.139420  6872 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.172456  6844 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.194782  6844 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.190136  6872 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.201413  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:20.215210  1566 leader_election.cc:336] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:20.216816  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:20.214432  6883 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.221693  6883 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.230620  6883 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:20.232118  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.239586   924 raft_consensus.cc:1275] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:20.244470  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.245487  1599 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:20.253161  1567 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.255347   968 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:20.261685  6875 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:20.262153  6875 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.262529  6875 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.237941  6801 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.263780  6844 consensus_queue.cc:1048] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:15:20.264386  6801 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.272909  1566 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.267376  6801 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.270215  6857 raft_consensus.cc:2804] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:20.277972  6857 raft_consensus.cc:697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.278923  6857 consensus_queue.cc:237] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.279587  1566 leader_election.cc:336] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.271023  6852 leader_election.cc:290] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.288160  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.288758  1599 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:15:20.294939  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:20.295682  1000 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:15:20.300027   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.301018   924 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:15:20.307582   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2840f4bbf21443318b9ad41dc7e3e853" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:20.308312   996 raft_consensus.cc:3060] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:20.314286  6872 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.315150   894 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:20.322253  6872 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.324538  6872 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.326318  6875 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.332708  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:20.334692   969 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.335225   969 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.335739  6852 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:20.336212  6852 raft_consensus.cc:493] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:15:20.338799  1566 leader_election.cc:336] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.342617  6874 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:20.343083  6874 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.343464  6874 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.343807  6877 raft_consensus.cc:2749] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:20.344929   969 leader_election.cc:336] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.336582  6852 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:20.347916   895 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:20.351783   969 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:20.352744  6874 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.355625   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:20.357272  1564 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.365036   996 raft_consensus.cc:2468] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:15:20.378377  1564 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:20.378799  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 46165.4us
I20260709 13:15:20.377331  6872 raft_consensus.cc:2804] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:20.400153   752 catalog_manager.cc:5697] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 13 to 14, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 14 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.408337  1567 leader_election.cc:304] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.410398  6874 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.411793  6875 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.383301  6857 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:20.420742  6872 raft_consensus.cc:697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.462281  6852 raft_consensus.cc:515] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.476352  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 68099.8us
I20260709 13:15:20.477119  1567 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:20.479977   895 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.482086  6856 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.482636  6856 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:20.487794  6885 raft_consensus.cc:2804] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:20.490031  6882 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.490644  6882 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.493563  6882 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.495677   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:20.497313  6852 leader_election.cc:290] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:20.500859  1566 leader_election.cc:336] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.501597  6880 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.502216  6880 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:20.506979   969 leader_election.cc:336] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.512008  6883 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.512655  6883 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.515162  6883 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.519702  6882 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.520506  6882 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.523317  6882 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.524194  6885 raft_consensus.cc:697] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.525418  6885 consensus_queue.cc:237] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.525977  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.532145  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:20.533161  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:20.556986   895 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.557515   895 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:20.558504  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:20.560040   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:20.560747   996 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:15:20.562273   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:20.459434  6857 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.572158  6882 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.573868  6878 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:20.574288  6878 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.574657  6878 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.578466  6875 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.579105  6875 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.567973  6856 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.583853  6862 raft_consensus.cc:2749] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:20.584816  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 107767us
W20260709 13:15:20.601806   969 leader_election.cc:336] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.609750  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:20.610621  1596 raft_consensus.cc:3060] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.616600   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6444551c84374ec98f4becf8b07446ba" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:20.599864  6857 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.617275   996 raft_consensus.cc:3060] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:20.619571   895 leader_election.cc:336] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.620321   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.623551  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.624190  1592 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:15:20.618111  6880 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.620916  6872 consensus_queue.cc:237] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.627130  1596 raft_consensus.cc:2468] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:15:20.639335  6892 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.640357  6869 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.639262  6862 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.636809   996 raft_consensus.cc:2468] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:20.647432  6862 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.648264  6892 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.644593  6869 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.642207  6882 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:20.649868  6874 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.650388  6882 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.650452  6874 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.650992  6882 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.651744  6862 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.652719  6874 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:20.612596   739 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 90752.5us
I20260709 13:15:20.653734   968 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.655186  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.655967  1567 leader_election.cc:304] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.657289   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.657863   924 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:20.658588   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.661682   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.662264   929 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:15:20.662590   968 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.664330  6860 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:20.664892  6860 raft_consensus.cc:493] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.668512  6860 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.668020  6871 raft_consensus.cc:2804] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:20.672446  1564 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:20.672922   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.673281  6889 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:20.673580   929 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 14.
I20260709 13:15:20.673771  6889 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.674229  6889 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:20.675352   895 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:20.675276  1566 leader_election.cc:336] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.676571  6871 raft_consensus.cc:697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:15:20.679912  1566 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:20.682787   969 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.671248  6869 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.677451  6871 consensus_queue.cc:237] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.692956  6892 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.660403  6862 raft_consensus.cc:2804] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:20.660933  6878 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:20.701905  1566 leader_election.cc:336] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.701022  6875 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.701443  6868 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.705683  6868 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.695048   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.707810  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.708302   929 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 7.
I20260709 13:15:20.708726  1592 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:20.707687  6892 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.709745  6878 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:20.709901  6892 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.710903   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b9f004a6f4293a658cca1bb92e319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:20.711490   996 raft_consensus.cc:3060] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.711968  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.712744  6852 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:20.713128  6852 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.713502  6852 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:20.715595   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.716216   929 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 11.
I20260709 13:15:20.715317   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:20.716825  6882 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.717214  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:20.719424  6860 raft_consensus.cc:515] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.721896  6860 leader_election.cc:290] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.722188  6889 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.723479  1596 raft_consensus.cc:2393] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 15: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:15:20.724097  1564 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:20.724841  6801 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:20.725230  6801 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
W20260709 13:15:20.725350  1566 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.725620  6801 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:20.716753   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.708029   926 raft_consensus.cc:2393] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 12: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:20.730654  6862 raft_consensus.cc:697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.731701  6862 consensus_queue.cc:237] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.708000  6868 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.735409  6852 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.735847  1566 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.742465  1566 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:20.712277  6892 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.737963   968 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.740276  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85bbb1b886474a538a2f1a32dd91b560" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:20.745370  1592 raft_consensus.cc:3060] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.741362  1000 raft_consensus.cc:1275] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:15:20.750030   996 raft_consensus.cc:2468] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 12.
I20260709 13:15:20.759896   896 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:20.767270   895 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.767974   895 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.769045  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:20.766330  6884 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:20.763134  6869 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.736887  6857 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.786392  6856 raft_consensus.cc:2804] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:20.809800   751 catalog_manager.cc:5697] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 15 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.790832  6853 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:20.817596  1592 raft_consensus.cc:2468] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:15:20.795469  6884 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.822674  6889 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.823077  6884 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.817605  6852 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.818778  1598 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 15: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:15:20.828539   969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111) [suppressed 294 similar messages]
W20260709 13:15:20.829433   969 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.829851   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:20.796108  6892 consensus_queue.cc:1048] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:15:20.821019  6801 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.832229   968 leader_election.cc:304] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:20.838260   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.838871   929 raft_consensus.cc:3060] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.839176  6880 raft_consensus.cc:2804] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:20.839743  6880 raft_consensus.cc:697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:20.840616  6880 consensus_queue.cc:237] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.842176  6857 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.819316  6856 raft_consensus.cc:697] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:20.843902  6856 consensus_queue.cc:237] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:20.847400   929 raft_consensus.cc:2468] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:20.834555   752 catalog_manager.cc:5697] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 12 to 13, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 13 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.849957   894 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.850698  6876 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.850342   752 catalog_manager.cc:5697] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 10 to 11, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 11 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.847378   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.852403   926 raft_consensus.cc:3060] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:20.852049   752 catalog_manager.cc:5697] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.855525   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.856350   929 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 15: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:20.857044  1564 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:20.858402  1566 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:20.847790  6860 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:20.866007  1566 leader_election.cc:336] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:20.866549  1566 leader_election.cc:336] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.867012  6857 raft_consensus.cc:2804] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:20.867280   969 leader_election.cc:336] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.868844  6801 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:20.872346  6884 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.873648  1002 raft_consensus.cc:2393] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 15: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
W20260709 13:15:20.866955   895 leader_election.cc:336] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.874804   895 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.877063   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.877640   929 raft_consensus.cc:3060] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.881349  6882 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.882790  6857 raft_consensus.cc:697] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.883791  6857 consensus_queue.cc:237] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.886659   895 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.891003   926 raft_consensus.cc:2468] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 8.
I20260709 13:15:20.892738  1567 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.895025  6852 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.898957  6884 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:20.904666  1564 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:20.907979  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:20.910463   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:20.911149   925 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:20.916394  6892 raft_consensus.cc:2804] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:20.917040  6892 raft_consensus.cc:697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.919430   925 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
I20260709 13:15:20.929409   929 raft_consensus.cc:2468] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
W20260709 13:15:20.927368  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.932798  6892 consensus_queue.cc:237] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.932423  1566 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:20.945303  6801 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.944620  1564 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:20.948626  6869 raft_consensus.cc:2804] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:20.953406  6869 raft_consensus.cc:697] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:20.960296  6869 consensus_queue.cc:237] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:20.963999 32224 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 876)
W20260709 13:15:20.970945  1566 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:20.977861  6902 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:20.978463  6902 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:20.980875  6902 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:21.003000  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:21.003767  1592 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:15:21.012384   894 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:21.023391   895 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:21.024741   969 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:21.015877  6856 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:21.050487  6856 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:21.051273  6856 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:21.066704   752 catalog_manager.cc:5697] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 11 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 12 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:21.085939  6852 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.098671  6856 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:21.103902  1598 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15.
W20260709 13:15:21.458420  6909 log.cc:927] Time spent T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
W20260709 13:15:21.460246  6908 log.cc:927] Time spent T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
W20260709 13:15:21.964555 32224 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 876)
W20260709 13:15:22.137652   784 debug-util.cc:398] Leaking SignalData structure 0x7b080031dd60 after lost signal to thread 32227
I20260709 13:15:22.225250   996 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33108 (request call id 1287) took 1118 ms. Trace:
I20260709 13:15:22.225921   996 rpcz_store.cc:276] 0709 13:15:21.106904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.107162 (+   258us) service_pool.cc:224] Handling call
0709 13:15:22.224597 (+1117435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.224819 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.224854 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.224901 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:22.224953 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.225016 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.225146 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:21.113690  6856 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.258476  6911 raft_consensus.cc:493] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.259061  6911 raft_consensus.cc:515] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.261579  6911 leader_election.cc:290] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.262125  6911 raft_consensus.cc:493] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.262629  6911 raft_consensus.cc:515] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.264950  6911 leader_election.cc:290] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.268118  6911 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.268605  6911 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.270596  6911 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.291658  1592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37802 (request call id 1328) took 1181 ms. Trace:
I20260709 13:15:22.296686  6913 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.296936  6856 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.297766   751 catalog_manager.cc:5697] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 15 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.299417  6856 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.300599   751 catalog_manager.cc:5697] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 7 to 8, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 8 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.301856  6856 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.298914  1592 rpcz_store.cc:276] 0709 13:15:21.109715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.109895 (+   180us) service_pool.cc:224] Handling call
0709 13:15:22.290855 (+1180960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.291120 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.291158 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.291225 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:22.291283 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.291357 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.291506 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.303960  6913 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.306635  6913 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.307078  6883 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.308516  6883 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.312897  6846 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.313516  6846 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.315639  6846 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.315901  6873 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.317291  6873 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.322103  6883 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.323146  6873 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.335541  6881 raft_consensus.cc:493] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.343883  6881 raft_consensus.cc:515] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.351155  6856 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.352782  6882 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.353652  6881 leader_election.cc:290] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.353348  6882 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.352021  6856 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.355474  6881 raft_consensus.cc:493] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.356238  6882 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.356731  6856 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.355997  6881 raft_consensus.cc:515] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.358587   752 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55314 (request call id 348) took 1345 ms. Trace:
I20260709 13:15:22.359740   752 rpcz_store.cc:276] 0709 13:15:21.013423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.013562 (+   139us) service_pool.cc:224] Handling call
0709 13:15:22.358523 (+1344961us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:21.072770 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:21.073268 (+   498us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:21.073299 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:15:21.073317 (+    18us) tablet.cc:662] Decoding operations
0709 13:15:21.073649 (+   332us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:21.073702 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:21.073727 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:21.073904 (+   177us) tablet.cc:701] Row locks acquired
0709 13:15:21.073921 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:15:21.074053 (+   132us) write_op.cc:270] Start()
0709 13:15:21.074161 (+   108us) write_op.cc:276] Timestamp: P: 1783602921074011 usec, L: 0
0709 13:15:21.074183 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:15:21.074750 (+   567us) log.cc:844] Serialized 538 byte log entry
0709 13:15:21.076568 (+  1818us) op_driver.cc:464] REPLICATION: finished
0709 13:15:21.076939 (+   371us) write_op.cc:301] APPLY: starting
0709 13:15:21.077011 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:21.077858 (+   847us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:21.077949 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:21.078740 (+   791us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:21.078891 (+   151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:21.078933 (+    42us) write_op.cc:312] APPLY: finished
0709 13:15:21.079108 (+   175us) log.cc:844] Serialized 25 byte log entry
0709 13:15:21.079260 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:21.079330 (+    70us) write_op.cc:454] Released schema lock
0709 13:15:21.079385 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:21.082840 (+  3455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:21.082868 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":201,"apply.run_cpu_time_us":2713,"apply.run_wall_time_us":2697,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4180,"num_ops":1,"prepare.queue_time_us":221,"prepare.run_cpu_time_us":2250,"prepare.run_wall_time_us":2251,"replication_time_us":2280}]]}
I20260709 13:15:22.393149  6911 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.393903  6911 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.394506  6883 raft_consensus.cc:493] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.374258  6881 leader_election.cc:290] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.398751  6876 raft_consensus.cc:493] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.402168  6881 raft_consensus.cc:493] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.402649  6876 raft_consensus.cc:515] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.402793  6881 raft_consensus.cc:515] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.405112  6876 leader_election.cc:290] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:22.411460   969 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:22.413631   969 leader_election.cc:336] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:22.415433   969 leader_election.cc:336] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:22.416903   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.416393  6880 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.396132  6883 raft_consensus.cc:515] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:22.419730   969 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:22.414245  6876 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
W20260709 13:15:22.421662   969 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.425688  6875 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.424470  6880 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.426287  6875 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.428073  6880 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.420300  6918 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.428344  6876 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.428794  6922 raft_consensus.cc:493] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.428714  6918 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.429387  6922 raft_consensus.cc:515] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.435041  6911 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:22.425164   969 leader_election.cc:336] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.436882  6844 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.442040   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.444363   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.444741   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.446050  6844 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.442893   999 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:15:22.454245   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.454891   996 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 17.
W20260709 13:15:22.455226   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.455922   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.456063  6875 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.443361  6913 raft_consensus.cc:493] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:15:22.456391   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.456758  6913 raft_consensus.cc:515] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.444393  6921 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.457736  6921 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.458721  6913 leader_election.cc:290] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.459455  6881 leader_election.cc:290] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.459959  6919 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.466526  6856 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:15:22.471776   969 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.467092  6856 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.464084  6919 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.473212  6920 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.478870  6920 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.481302  6920 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.453240  6844 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.482633  6856 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.483265  6856 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.449154  6880 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:15:22.484268   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.484201  6880 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.485306  6881 raft_consensus.cc:493] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.485857  6881 raft_consensus.cc:515] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.478197  6919 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.486676  6880 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.494058   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4018887e34a40108555fd3cbaa2c975" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:22.493417  6916 log.cc:927] Time spent T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:15:22.483816  6856 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.496418  6921 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.497598  6883 leader_election.cc:290] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.509194  6856 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:22.500335  1598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37802 (request call id 1324) took 1605 ms (client timeout 1680 ms). Trace:
W20260709 13:15:22.509999  1598 rpcz_store.cc:269] 0709 13:15:20.895025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.907743 (+ 12718us) service_pool.cc:224] Handling call
0709 13:15:22.500114 (+1592371us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54007dd2d4. stack: 00007f0cffa20343 00007f0cffa20240 00007f0cff3af1d4 00007f0d090ed304 00007f0d090ed299 00007f0d02aa8571 00007f0d02b432fa 00007f0d07b04b17 00007f0d0107f6d4 00007f0d0107f662 00007f0d0107f591 00007f0d0107f50c 00007f0d0107e7c2 00007f0d00e7a48c 00007f0d00e798c6 00007f0d00e7926f
0709 13:15:22.500243 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409646464}
I20260709 13:15:22.502955  1003 raft_consensus.cc:1275] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:15:22.504159  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdca38d5137a4409b2e61eaf2e13fd7b" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:22.513993   784 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd020 after lost signal to thread 787
I20260709 13:15:22.514755   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.516247  6847 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:22.516672  6847 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.517045  6847 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:22.518093   896 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.520709  6917 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:22.521133  6917 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.521461  6917 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:22.510672   969 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.512367  6881 leader_election.cc:290] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.524660  6882 consensus_queue.cc:1048] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:22.527894   969 leader_election.cc:336] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.538193  6918 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.538899  6856 raft_consensus.cc:493] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.539484  6856 raft_consensus.cc:515] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.539793  6876 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:22.540835   784 debug-util.cc:398] Leaking SignalData structure 0x7b08000966c0 after lost signal to thread 954
W20260709 13:15:22.541793   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.544239  6912 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.544811  6912 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.549556  6918 raft_consensus.cc:493] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.552229  6918 raft_consensus.cc:515] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.553443  6917 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.554589  6918 leader_election.cc:290] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.555692  6856 leader_election.cc:290] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.557216  6912 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.558439  6847 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.560340   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.561055   923 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 8.
I20260709 13:15:22.571911   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155790402004e4f930983841dfd8c64" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.577265   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.577659   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.577809   752 catalog_manager.cc:5697] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 15 to 16, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 16 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:22.584856  1001 raft_consensus.cc:1275] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:15:22.566535  6912 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.580952  6856 raft_consensus.cc:493] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.596845   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.598728  6874 raft_consensus.cc:493] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.599449  6874 raft_consensus.cc:515] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:22.600594   784 debug-util.cc:398] Leaking SignalData structure 0x7b080038b7c0 after lost signal to thread 1028
I20260709 13:15:22.583747  6876 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.605003  6860 raft_consensus.cc:493] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.608373  6922 leader_election.cc:290] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.587957  6882 raft_consensus.cc:493] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.615906   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248dca75dd144af2a2106e703fe92de4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.612284  6922 raft_consensus.cc:493] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.613265  6856 raft_consensus.cc:515] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:22.595487  6913 raft_consensus.cc:493] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.617828  6922 raft_consensus.cc:515] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.624866  6913 raft_consensus.cc:515] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.630880   925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.632411   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.615864  6882 raft_consensus.cc:515] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.633986   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.612640  6876 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.636654   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.637342   929 raft_consensus.cc:2393] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 18: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
W20260709 13:15:22.636341   969 leader_election.cc:336] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.638448   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef39166ba76a41d18b3770b743ea3488" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.640673   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.641278   924 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:22.606475  6874 leader_election.cc:290] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.592171  6912 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.638731  6847 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.612931  6860 raft_consensus.cc:515] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.643067  6876 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.643560  6912 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.644398  6860 leader_election.cc:290] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:22.645164   784 debug-util.cc:398] Leaking SignalData structure 0x7b080033a620 after lost signal to thread 1625
W20260709 13:15:22.645493   784 debug-util.cc:398] Leaking SignalData structure 0x7b080033a400 after lost signal to thread 1626
I20260709 13:15:22.651798  6917 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.652554  6922 leader_election.cc:290] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.655088   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.655408   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.656059   927 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:22.656987   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:22.659823   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 48861.2us
I20260709 13:15:22.661350  1000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b38ec4f420f4de9883306887a006c24" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.662007  1002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.665678  1001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdfea09a06ff4100a0b80619f35f7008" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.662587  1002 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 8.
I20260709 13:15:22.666570   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.667568   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfb3f7cad745918e1bc1faf5cc697c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.669225  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379c9f3a933409ca08bceb325f43132" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.669960   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6874f81648724ea980572af4b6fa79d2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.672640   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:22.675063   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.675180   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.676334  6801 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.680367   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:22.680987   996 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:22.679958  6801 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.691924  6801 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.692489  6801 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.692560  6928 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:15:22.693367   969 leader_election.cc:336] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.693220  6928 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.694538  6801 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.695894   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.696692  6923 raft_consensus.cc:493] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.681576   896 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.687680   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.667644  6922 raft_consensus.cc:493] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.706009   998 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:15:22.695408  6928 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.720400   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.721704   967 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.724581   996 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:15:22.724511  6917 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:22.727867  6917 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.728281  6917 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:22.727880   896 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.731873  6877 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:22.732893   967 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.720165  6892 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.734153  6892 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.731055  6921 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:22.736159  6892 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.736650  6920 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:22.739715  6801 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.743502   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.744673   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.745982   967 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:22.746181  6873 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:22.746948  6873 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:22.747311  6873 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.747700  6873 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:22.748328  6921 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:22.748764  6921 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.749118  6921 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:22.750892   895 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:22.751658   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.752101   895 consensus_peers.cc:597] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:22.752472   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.710783  6923 raft_consensus.cc:515] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.754536  6913 leader_election.cc:290] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.755121  6882 leader_election.cc:290] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.722491  6918 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:22.752925   895 leader_election.cc:336] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.760000  6918 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.760195   895 leader_election.cc:304] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.760651  6918 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:22.708653  6922 raft_consensus.cc:515] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:22.766583   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.736063  6856 leader_election.cc:290] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:22.788374   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 60589.6us
I20260709 13:15:22.789120   896 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:22.792824   895 leader_election.cc:336] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:22.793658   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.794175   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.794339  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 370264us
W20260709 13:15:22.794680   895 leader_election.cc:336] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.795861  6856 raft_consensus.cc:2749] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:22.795276   895 leader_election.cc:304] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.797487  6876 raft_consensus.cc:2749] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:22.797526   895 leader_election.cc:336] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.798208   895 leader_election.cc:304] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:22.799535   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.800025   895 leader_election.cc:336] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.800420   895 leader_election.cc:304] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:22.801589   895 leader_election.cc:336] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.802562   923 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:22.802989   895 leader_election.cc:304] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.804329  6928 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.804838  6928 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.802398  6874 raft_consensus.cc:2749] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:22.798877  6847 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:22.806841  6928 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.806906  6847 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:22.807442  6881 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:22.807437  6847 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:22.810155   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.810243   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01fc410be7f74515adce264a85731b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.809700  6922 leader_election.cc:290] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.755170  6923 leader_election.cc:290] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.811447   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.812216   927 raft_consensus.cc:1275] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:22.813119  6876 raft_consensus.cc:2749] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:22.817485  6847 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.819403  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4d7e87915947148833db3fe84cf623" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.821835  6881 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:15:22.831753   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:22.832114  6856 raft_consensus.cc:2749] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:22.832322   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:22.835435   895 leader_election.cc:336] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.838582   895 leader_election.cc:304] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.841640  6881 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.843230   923 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:22.844828   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
W20260709 13:15:22.843825   895 leader_election.cc:336] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:22.845544   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:22.846383   895 leader_election.cc:304] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.849479  6925 raft_consensus.cc:2749] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:22.849584  6869 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.849507  6939 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.851064  6939 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.847915  6856 raft_consensus.cc:2749] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:22.849756  6936 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.853343   751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41676 (request call id 227) took 1823 ms. Trace:
I20260709 13:15:22.852615  6936 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.860638  6868 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.861210  6868 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.865787  1598 raft_consensus.cc:1275] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:15:22.867619  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a3708334cc4bd59c73d494d2747d54" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.868186  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d4a96b7f094cf691b31dbe6fb015bd" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.868119  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.886948   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:22.850301  6869 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.853785   751 rpcz_store.cc:276] 0709 13:15:21.030354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.030582 (+   228us) service_pool.cc:224] Handling call
0709 13:15:22.853275 (+1822693us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:22.320536 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:22.321212 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:22.321250 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:15:22.321269 (+    19us) tablet.cc:662] Decoding operations
0709 13:15:22.321650 (+   381us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:22.321722 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:22.321763 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:22.322134 (+   371us) tablet.cc:701] Row locks acquired
0709 13:15:22.322153 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:22.322284 (+   131us) write_op.cc:270] Start()
0709 13:15:22.322391 (+   107us) write_op.cc:276] Timestamp: P: 1783602922322239 usec, L: 0
0709 13:15:22.322415 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:15:22.323167 (+   752us) log.cc:844] Serialized 913 byte log entry
0709 13:15:22.600398 (+277231us) op_driver.cc:464] REPLICATION: finished
0709 13:15:22.767484 (+167086us) write_op.cc:301] APPLY: starting
0709 13:15:22.767641 (+   157us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:22.769942 (+  2301us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:22.770094 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:22.821553 (+ 51459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:22.831707 (+ 10154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:15:22.831754 (+    47us) write_op.cc:312] APPLY: finished
0709 13:15:22.831993 (+   239us) log.cc:844] Serialized 33 byte log entry
0709 13:15:22.837352 (+  5359us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:22.840578 (+  3226us) write_op.cc:454] Released schema lock
0709 13:15:22.841192 (+   614us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":120659,"apply.run_cpu_time_us":5830,"apply.run_wall_time_us":122581,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12487,"num_ops":2,"prepare.queue_time_us":17045,"prepare.run_cpu_time_us":32088,"prepare.run_wall_time_us":92634,"raft.queue_time_us":95550,"raft.run_cpu_time_us":23060,"raft.run_wall_time_us":42093,"replication_time_us":277849,"thread_start_us":219175,"threads_started":4,"wal-append.queue_time_us":112972}]]}
I20260709 13:15:22.889964  6869 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.891764  6847 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.892446  6936 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.894860  6939 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.893684  6868 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.893141  6921 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.899782  6923 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.897344  6852 raft_consensus.cc:493] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.914762  6852 raft_consensus.cc:515] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.915912  6923 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.916867  6852 leader_election.cc:290] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.900995  6873 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.902397  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.904089  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.904320  1594 raft_consensus.cc:1275] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:22.917694  1599 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 18.
I20260709 13:15:22.917783  1600 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:15:22.918465  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0cca622722b48a7ba611a3b77376f80" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.918648  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e1447d2c0c4822aac226064c087c42" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.918736  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.919577  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.920214  1594 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:15:22.920377  6939 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64f2cb5b0e174002811ef197f9eec829)
I20260709 13:15:22.904623  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.904830  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.920912  6923 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.922132  1593 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 9.
I20260709 13:15:22.905647  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.922576  6918 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.924649  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.925110  6873 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.908608  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.926244  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91555b4ec53349c287c5221b9030c23a" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.928020  6917 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.928534  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.929991  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.841619  6937 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:22.920257  6863 raft_consensus.cc:493] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.905588  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.921034  6862 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.907801  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.920902  6939 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.936892   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf198f3795b43ff951a11167bc21580" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.937036  6941 consensus_queue.cc:1048] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:22.937413  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65c03394bf04e2fb03562b27d85f5b3" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.938697  6939 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.939345  1600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.939921  1600 raft_consensus.cc:3060] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:22.942421  6862 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.943859  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:22.944609  1004 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 9: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:22.945386  6862 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:22.945219  6863 raft_consensus.cc:515] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.923718  6852 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:22.924018  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.934377   996 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 15: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:22.947917  6873 consensus_queue.cc:1048] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:22.948539  1597 raft_consensus.cc:2376] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 15. Re-sending same reply.
I20260709 13:15:22.949447  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.950443  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c43ab43025004edcacdb3d8a30106319" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.947363  6863 leader_election.cc:290] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:22.932724  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.952172   998 raft_consensus.cc:1275] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:22.952739  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.934788  1592 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 14.
I20260709 13:15:22.954406   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.954644   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.954960  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d5a097d0fe49bab1a3ec1f78f4164d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.953851  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cfaab49f56546a49978c6055f57640f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.956934  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.951469  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de58b0d5ad94d869fac3b0491143543" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.935403  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.921196  1598 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
I20260709 13:15:22.947873  6852 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:22.959424  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.956991   968 leader_election.cc:304] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.960335  1600 raft_consensus.cc:2468] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:15:22.961210  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b871863269841f7b0aa8baf5e640865" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.964053   996 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:22.963953   968 leader_election.cc:304] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.964478  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.965399   997 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
W20260709 13:15:22.966137 32224 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 876)
I20260709 13:15:22.964638  6911 raft_consensus.cc:2749] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:22.966883  6883 raft_consensus.cc:2749] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:22.968565  1595 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:15:22.969255   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe55b34a7fd4408b9a1e51ded1fd129" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.974479   996 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:22.930059  6917 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.975301   999 raft_consensus.cc:1240] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:22.977818  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 81488.6us
I20260709 13:15:22.960397  6918 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:22.980386  1601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f44707b1d3466e91403f063187a9b2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:22.981902   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.983477  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.984405  1599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.985731  1592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.986864  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:22.988812   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.991993   968 leader_election.cc:304] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.992882   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.994789   968 leader_election.cc:304] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:22.999018   750 catalog_manager.cc:5697] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 12, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:23.007063   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:22.998945  6911 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:23.010854   927 raft_consensus.cc:1275] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:15:23.011010  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.010896   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "633fa6f0c6434c49aeb87667c9d97141" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.012096  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb3d540c3fc49bf926540f7bd349110" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.012419   968 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.014178   968 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:22.960705  6937 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.016294  6880 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:23.016613  6918 raft_consensus.cc:2749] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:23.018107   968 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.019647   929 raft_consensus.cc:1275] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:23.023698   924 raft_consensus.cc:1240] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:23.003844  6883 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:23.020579   927 raft_consensus.cc:1240] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:23.024279   997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.021848   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.025357   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.027920   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.034749   968 leader_election.cc:304] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.000135  6806 consensus_queue.cc:1048] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:15:23.038166   926 raft_consensus.cc:1275] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:15:23.038944   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.011008  6911 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.039155   968 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.038298  6860 raft_consensus.cc:2749] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:23.017729  6912 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:23.044757   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedf1cefa52d44d884e14c5261c6437a" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.041774   968 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.040385  6911 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.037019  6883 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.036383  6873 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:23.052503   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.053153   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd817716cdc4448b3f884eecc94351c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.055197   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.056372   924 raft_consensus.cc:2393] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 15: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:23.043049  6844 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.061156   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731bda621b364aaa81d868d0cef5e739" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.062076  6877 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:23.063324  6877 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.063740  6877 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:23.064654  1004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad6ca2a4a8134a10b60d7308b9a660db" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.056195  6873 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.076951  6873 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:23.056322  6883 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:23.065603  6863 consensus_queue.cc:1059] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:15:23.067337  6935 log.cc:927] Time spent T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:15:23.071156  6911 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.072750  6852 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:23.073529  6921 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.060400  6920 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:23.045514  6912 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.091815   968 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.091971   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.098596  6937 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:23.101583  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.184453  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 82928.7us
I20260709 13:15:23.185492  6923 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:23.186213  6923 consensus_queue.cc:1059] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:23.187288  6863 raft_consensus.cc:3055] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:23.187662  6863 raft_consensus.cc:740] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:23.187852  6852 consensus_queue.cc:1059] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:23.188601  6863 consensus_queue.cc:260] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.189620  6863 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:23.189742  1567 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.190550  6936 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:23.106169  1564 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.192204  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 86091.1us
W20260709 13:15:23.109018   895 leader_election.cc:336] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.116497   894 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.193081  6936 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:23.193173   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 84149.6us
W20260709 13:15:23.193661   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 77250.2us
W20260709 13:15:23.193734   895 leader_election.cc:336] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.194412   894 leader_election.cc:304] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.193866  6936 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:23.195770  6847 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:23.195926  6885 consensus_queue.cc:1048] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:23.196507  6847 raft_consensus.cc:2749] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:23.196982  6801 consensus_queue.cc:1048] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:23.104715  6939 consensus_queue.cc:1048] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:15:23.195765  6934 consensus_queue.cc:1059] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:15:23.108671  6862 consensus_queue.cc:1048] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:15:23.108392  6920 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:23.199483  6920 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.200083  6939 raft_consensus.cc:3055] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:23.200510  6939 raft_consensus.cc:740] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:23.201313  6939 consensus_queue.cc:260] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.202471  6939 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:23.104573  6912 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.205806   894 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.207228   894 leader_election.cc:304] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.136938  6911 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.207849  6950 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.208254  6882 raft_consensus.cc:2749] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:23.154965   751 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 15, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 15 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:23.165228  6883 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.212170  6883 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 19 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.212790  6862 consensus_queue.cc:1059] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:23.213529  6862 consensus_queue.cc:1243] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer 91ff541db50a49efa03669dfd777fc8e log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:23.213994  6862 consensus_queue.cc:1048] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:23.172685  6919 raft_consensus.cc:2749] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.217597   894 leader_election.cc:304] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.224499  6947 raft_consensus.cc:2749] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:23.224951  6912 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.226599   894 leader_election.cc:304] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.246737  6936 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.142135  6873 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.189078  6941 raft_consensus.cc:3055] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:23.247732  6941 raft_consensus.cc:740] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:23.247398  6936 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.248661  6941 consensus_queue.cc:260] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.249752  6936 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:23.249850  6941 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:23.247401  6947 raft_consensus.cc:2749] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:23.194535   895 leader_election.cc:336] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:23.251842   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 57326.6us
W20260709 13:15:23.252969   895 leader_election.cc:336] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.253381   895 leader_election.cc:304] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.197366  6885 raft_consensus.cc:3055] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:23.255628  6885 raft_consensus.cc:740] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:23.256525  6885 consensus_queue.cc:260] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.257886  6885 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:23.135542  6868 raft_consensus.cc:3055] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:23.261253  6868 raft_consensus.cc:740] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:23.262096  6868 consensus_queue.cc:260] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.263334  6868 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:23.268354   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.269131   924 raft_consensus.cc:2393] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 15: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:23.271021  6950 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.271651  6950 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.280478  6950 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.283416   998 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:23.283955   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.284974  6950 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:23.285498   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.287076   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.288612   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.150713  6877 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.107832   996 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:23.155066  6880 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:23.216627  1567 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.291522  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 74969.4us
I20260709 13:15:23.292269  1567 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.293758  6863 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:23.293864  1567 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.231560  6920 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:15:23.232538   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 175336us
I20260709 13:15:23.294489  6863 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:23.294716  6920 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:23.295200  6920 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.300117  6863 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:23.302143  1567 leader_election.cc:304] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.303133  6863 raft_consensus.cc:2749] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:23.116592   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:23.304548   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 187977us
I20260709 13:15:23.311990  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.313059  6863 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.316883   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.318493   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d548ef2981894fe6972da35c997f5cff" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.320518  6877 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.124366  6917 raft_consensus.cc:493] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.321269  6917 raft_consensus.cc:515] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.266355  6882 raft_consensus.cc:2749] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:23.267941   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.324127   927 raft_consensus.cc:2393] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 9: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:23.323364  6917 leader_election.cc:290] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.320605  1004 raft_consensus.cc:1240] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:23.326848   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.236223  6912 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.329821  6873 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.318486  6947 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:15:23.246970   894 leader_election.cc:304] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.334919  6947 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:23.335292  6917 raft_consensus.cc:2749] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:23.335335  6947 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:15:23.336321   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 89429.1us
I20260709 13:15:23.336385  6947 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.337167   894 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.337772  6947 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.337986  6917 raft_consensus.cc:2749] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:23.338948   894 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:23.340760   895 leader_election.cc:336] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:23.407305   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66580.4us
I20260709 13:15:23.311255  6920 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.413318  6920 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.343467  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c72e2f13b3404583a1c34e15098c42b9" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.347908  1004 raft_consensus.cc:1240] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:23.352837  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.354226  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.416405  6917 consensus_queue.cc:1059] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:23.417459  6917 raft_consensus.cc:3055] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:23.417783  6917 raft_consensus.cc:740] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:23.418615  6917 consensus_queue.cc:260] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.422276  6917 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:23.425633   967 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.426496  6877 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.431406  6882 raft_consensus.cc:493] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.432027  6882 raft_consensus.cc:515] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.434185  6882 leader_election.cc:290] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.438875   967 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.440091  6920 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:15:23.441243   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 102367us
I20260709 13:15:23.442443   894 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.369343  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.448068  6917 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:23.373324  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.443069  6882 raft_consensus.cc:2804] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:23.449350  6882 raft_consensus.cc:697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:23.450349  6882 consensus_queue.cc:237] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.452524   894 leader_election.cc:304] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.453380  6917 raft_consensus.cc:2749] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:23.458892  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:23.458540   968 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.460956  6873 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:23.462188   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.462955  6873 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:23.464972   968 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.445158  1596 raft_consensus.cc:2468] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:23.372676  1598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
W20260709 13:15:23.368402  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75155.3us
I20260709 13:15:23.480700   924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:23.482524   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:15:23.504637   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 49963.5us
W20260709 13:15:23.537039   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 72136.7us
I20260709 13:15:23.608372  6873 raft_consensus.cc:2749] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:15:23.609479   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 71531.5us
I20260709 13:15:23.610545   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.482569   895 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.620882   894 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.621850  6917 raft_consensus.cc:2749] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.623113   894 leader_election.cc:304] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.624552  6917 raft_consensus.cc:2749] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:23.615334   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 132802us
I20260709 13:15:23.631939  6885 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:23.632332   895 leader_election.cc:336] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.632586  6885 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.639786  6885 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:23.646621  1566 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.647143  1566 leader_election.cc:336] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.648257   967 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.649505   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:23.665936  6918 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:15:23.667116   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 56641.1us
I20260709 13:15:23.667882   968 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.671252  6877 raft_consensus.cc:2804] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:23.671849  6877 raft_consensus.cc:697] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:23.672705  6877 consensus_queue.cc:237] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.676290  6844 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:23.680747   967 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.681572  6844 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:23.685576   894 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:23.686645  6917 raft_consensus.cc:2749] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:23.688462  1564 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.689936  6885 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:23.703269  1565 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.704377  1004 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:23.710328  1596 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:15:23.711977  6844 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:23.716579  6947 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:23.722577  1004 raft_consensus.cc:1240] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:23.647495  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.724586  6941 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:23.729152  6913 consensus_queue.cc:1059] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:23.730640  6913 raft_consensus.cc:3055] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:15:23.730932  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 83804.5us
W20260709 13:15:23.731752  1566 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.732144  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.732903  6941 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:23.733891  1566 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:23.731019  6913 raft_consensus.cc:740] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:23.736385  6913 consensus_queue.cc:260] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.737951  6913 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:23.744496  6882 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:23.744685  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:23.745592  1596 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:15:23.745134  6882 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:23.746924  6882 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.748281  6882 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.748517  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701b27559f8e47f9b63e5d220d3d5a58" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:23.753943   751 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 15 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:23.768209   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:23.773622   895 leader_election.cc:336] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:23.775094  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:23.777778  1598 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:23.779585  1594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3247a08a45747539334328f4abf2acc" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:23.790412  1566 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.794999  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.796126  1566 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.796630   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:23.796814  1566 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.797721  1566 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.798106  1566 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:23.797864   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:23.799301  6941 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:23.800549  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.801661  1566 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.802131  1566 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:23.797039  6868 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:23.804157  1566 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:23.803380  6863 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.808358  1566 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:23.854411   895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 66846.6us
W20260709 13:15:23.858868   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 70815.6us
W20260709 13:15:23.861532   896 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.865185   923 raft_consensus.cc:1240] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:23.886466   922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.893414  6882 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.894155   922 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:23.894593  6882 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.869824   923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:23.898340   923 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:23.921397  1004 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:23.923960  6947 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:15:23.925947  6917 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:23.926772  6917 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:23.929466  6917 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.935561  6917 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:23.937505  6882 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:23.939576   996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:23.948192   894 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:23.950986   894 leader_election.cc:304] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:23.961297   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:23.955428  6882 raft_consensus.cc:2749] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:23.966138  6847 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:23.966723 32224 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 876)
I20260709 13:15:23.967206  6937 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:23.972047   969 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.983182   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 113305us
I20260709 13:15:23.974167  6847 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.008625   922 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:15:24.009835  6856 raft_consensus.cc:493] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 597687e464174d1ea6043161e037d240)
I20260709 13:15:24.010577  6856 raft_consensus.cc:515] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:23.998678  6847 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.038975  6856 leader_election.cc:290] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:24.049582  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 245277us
I20260709 13:15:24.061139   894 leader_election.cc:304] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:24.062690   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 63836.2us
I20260709 13:15:24.063944  6970 raft_consensus.cc:2749] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:24.064860  6970 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:15:24.069133   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:24.069720  1566 leader_election.cc:336] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.070475  1566 leader_election.cc:304] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:24.072422  1566 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.073005  1566 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:24.073889  6937 raft_consensus.cc:2749] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:24.074772  6937 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:24.076577  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.089279  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:24.091712  1595 raft_consensus.cc:1275] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:15:24.093037  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4997f2c7b33048a5b010a4e0c7866fa2" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:24.097427  6847 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:24.100397   969 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:24.097466  6911 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:15:24.112520  1564 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:24.119802  6974 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:24.120075   998 raft_consensus.cc:1240] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Rejecting Update request from peer 91ff541db50a49efa03669dfd777fc8e for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:24.125028  1593 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:24.129599  6882 consensus_queue.cc:1059] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:24.130890  6882 raft_consensus.cc:3055] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:24.131304  6882 raft_consensus.cc:740] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:24.121938  6974 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.133534  6974 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.133765  6882 consensus_queue.cc:260] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.135320  6882 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:24.147173  6877 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:15:24.153506   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:24.192823   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:24.204775   894 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:24.184088  6847 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:24.221082   895 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.214262  6917 consensus_queue.cc:1048] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
W20260709 13:15:24.239233   895 leader_election.cc:336] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.239881   895 leader_election.cc:304] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:24.273525  6974 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.277432  6974 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:24.282132   895 consensus_peers.cc:597] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:24.283666   895 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.287853   928 raft_consensus.cc:3060] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:24.282132  6856 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:24.309052   752 catalog_manager.cc:5697] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 15, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 15 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:24.373617   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
I20260709 13:15:24.374413   998 raft_consensus.cc:606] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Received request to transfer leadership
I20260709 13:15:24.383919   928 raft_consensus.cc:1275] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:15:24.388885   752 catalog_manager.cc:5697] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 12 to 13, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } } }
I20260709 13:15:24.392590  6967 raft_consensus.cc:2749] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:24.399051   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
I20260709 13:15:24.399858   998 raft_consensus.cc:606] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Received request to transfer leadership
I20260709 13:15:24.398919  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:24.404995  1593 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 16: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
W20260709 13:15:24.417392   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 80206.1us
I20260709 13:15:24.427332  6918 raft_consensus.cc:993] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: : Instructing follower 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:24.427971  6918 raft_consensus.cc:1081] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Signalling peer 91ff541db50a49efa03669dfd777fc8e to start an election
I20260709 13:15:24.445166   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:24.446046   929 raft_consensus.cc:2393] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 16: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:24.458945  6873 consensus_queue.cc:1048] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:15:24.458302   894 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:24.465333  6917 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:15:24.477703   740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 44966.2us
I20260709 13:15:24.479373   998 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "3bb5726887244caca146e176cc6bd070"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51112
W20260709 13:15:24.501016  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 43517.2us
W20260709 13:15:24.501737  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 65417.3us
I20260709 13:15:24.524726   998 raft_consensus.cc:606] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Received request to transfer leadership
W20260709 13:15:24.552520   801 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:15:24.553021   801 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 5b9d8e1d04724f1ba621ecf90cc49b82 already in progress
W20260709 13:15:24.553335   801 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 5b9d8e1d04724f1ba621ecf90cc49b82 already in progress
W20260709 13:15:24.574368  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.579635  1566 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:24.581338  6868 raft_consensus.cc:2749] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:15:24.592559   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:24.593362  1564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 77117.1us
I20260709 13:15:24.600915   928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82"
dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
 from {username='slave'} at 127.0.0.1:42382
I20260709 13:15:24.601554   928 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:24.601917   928 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.608170  6868 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.608773  6868 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.610949  6868 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.612944   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:24.613672   926 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 18.
I20260709 13:15:24.612949   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:24.616842  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.615814   998 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 18.
I20260709 13:15:24.617574  6937 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:24.617980  6937 raft_consensus.cc:493] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.618301  6937 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:24.634114  6937 raft_consensus.cc:515] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.636284  6937 leader_election.cc:290] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:24.596079  1566 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.639526  1566 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:24.639962   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
W20260709 13:15:24.640672  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 44626.8us
W20260709 13:15:24.641201  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:24.640681   930 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:24.637761   928 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.644129   926 raft_consensus.cc:1240] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Rejecting Update request from peer 3bb5726887244caca146e176cc6bd070 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:24.648937   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:24.649470   998 raft_consensus.cc:3055] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:24.649727   998 raft_consensus.cc:740] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:24.650534   998 consensus_queue.cc:260] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.652114   928 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:24.652318  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a8781662d54452a1478c2ae0e3f911" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:24.652980  1003 raft_consensus.cc:3060] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:24.654405   930 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 19.
I20260709 13:15:24.655578  1564 leader_election.cc:304] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.657079  6996 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:24.657819  6996 raft_consensus.cc:2804] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:24.657929  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:24.658572  1596 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.658658   998 raft_consensus.cc:3060] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.658823  6996 raft_consensus.cc:697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 19 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:24.660058  6996 consensus_queue.cc:237] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.668190  1003 raft_consensus.cc:2468] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 19.
I20260709 13:15:24.679661  6868 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.680425  6868 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.682782  6868 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.684706   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:24.685511   930 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:15:24.675513   998 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:15:24.686818  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.687644  6863 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:24.688019  6863 raft_consensus.cc:493] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.688356  6863 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:24.692144  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:24.695053   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:24.695758   998 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 9.
I20260709 13:15:24.696478  1596 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:15:24.700353  6863 raft_consensus.cc:515] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.708657   894 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.710969  6967 raft_consensus.cc:2804] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:24.713694  6967 raft_consensus.cc:697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:24.714831  6967 consensus_queue.cc:237] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.736320   752 catalog_manager.cc:5697] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:24.737857  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:24.746626   895 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:24.765508   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:24.766105   926 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:24.779078   926 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
I20260709 13:15:24.780175  1564 leader_election.cc:304] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.780917  6868 raft_consensus.cc:2804] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:24.781649  6868 raft_consensus.cc:697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:24.780917  6863 leader_election.cc:290] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 10 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.783852  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9794c08fc6b2428b981acd6c3c238b5f" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:24.784868  1003 raft_consensus.cc:3060] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:24.785351   752 catalog_manager.cc:5697] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 15 to 16, leader changed from 3bb5726887244caca146e176cc6bd070 (127.31.120.3) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:24.782971  6868 consensus_queue.cc:237] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.795369  1003 raft_consensus.cc:2468] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 10.
W20260709 13:15:24.804322   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:24.824260  6996 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.824923  6996 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.827332  6996 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.828220   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:24.829659   930 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:24.831403  1564 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.833384  6863 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:24.833827  6863 raft_consensus.cc:493] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.834676  6863 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.842430   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:24.843186   998 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:24.844146   752 catalog_manager.cc:5697] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:24.854139  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:24.889528  6863 raft_consensus.cc:515] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.893188   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:24.893836   930 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.895999   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8772f5977d4b4dc6b9f7f5dcf7019b90" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:24.896734   998 raft_consensus.cc:3060] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.910174   998 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:15:24.920660  1567 leader_election.cc:304] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:24.921682  6996 raft_consensus.cc:2804] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:24.922245  6996 raft_consensus.cc:697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 16 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:24.923259  6996 consensus_queue.cc:237] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.927100  6868 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:24.927699  6868 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:24.929926  6868 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.931833   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:24.932880   926 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:24.912599   930 raft_consensus.cc:2468] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:15:24.933828  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:24.913146  6863 leader_election.cc:290] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:24.934746  1003 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:24.937844  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:24.941762  6863 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:24.942142  6863 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:24.942503  6863 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:24.947501   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:24.953006   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:24.958607   752 catalog_manager.cc:5697] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:24.974588 32224 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 876)
W20260709 13:15:24.994412  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:25.004249  6863 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.007323   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.012457  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.013381  1003 raft_consensus.cc:2393] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.020416   930 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:25.017736  6863 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.023063  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:15:25.025254  6996 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:25.025758  6996 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:25.026669  6996 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.028903  6947 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.043160  6947 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.049028  6947 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.049954   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:25.050609   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.050668   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 15.
I20260709 13:15:25.052515   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.056427  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.057096  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 14.
I20260709 13:15:25.059118  6856 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:25.060029  6856 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.060441  6856 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.074532  6856 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.078817   930 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:25.082763  6996 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:25.101697  6856 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.102666  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.102721   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.103379  1593 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.103377   752 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.103469   998 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.125607   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:15:25.126840   896 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.127949  6947 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:25.128461  6947 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:25.129379  6947 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.145329   895 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:25.148573  1593 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 16.
I20260709 13:15:25.166534   751 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.175395   998 raft_consensus.cc:1275] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:25.176964  6937 consensus_queue.cc:1048] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:25.189400   930 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:25.190912  6868 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:25.214175   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:25.214723   969 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.239480   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.247630  1003 raft_consensus.cc:1275] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:15:25.249719   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.252131  6868 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.252806  6868 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.254990  6868 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.256711   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.257481   926 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:25.259269  1564 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.260398  6868 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:25.260903  6868 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.261372  6868 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:25.266482  6996 consensus_queue.cc:1048] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:25.266840   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:25.267439   969 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.267922   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.276273  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:25.277089  1003 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.287864  6868 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.297374  6868 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.298834   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.298748   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.306419   998 raft_consensus.cc:2393] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.322172   999 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:25.323796  6996 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:25.338043   926 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:25.339534  1564 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:15:25.350715  6868 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:25.351303  6868 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:25.356379  6868 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.360641  1593 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:15:25.364632  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:25.370625  6919 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.371169  6919 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.375635  6967 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:25.375094  6856 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.377961  6856 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.381084  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:25.382015  1003 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 10.
I20260709 13:15:25.383559   896 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.384684  6967 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:25.385165  6967 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:25.385322  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:25.385696  6967 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:25.393796  6856 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.396575  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.397257  1596 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 9.
I20260709 13:15:25.401688   930 raft_consensus.cc:1275] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:25.404634  6885 consensus_queue.cc:1048] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:25.374773   998 raft_consensus.cc:1275] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:25.409132  6856 consensus_queue.cc:1048] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:25.423983   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.424700   926 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:25.433753   751 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 11 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 12 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.440745  6920 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.441448  6920 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.444067  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.444878  6919 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.445528  6920 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.446821  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.453682  6967 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.472576  6967 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:25.473920   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 55840.5us
I20260709 13:15:25.475276  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.476001  1593 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:15:25.494884   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.513305   998 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:25.516028  1566 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.529896   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.529924  1593 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:15:25.555761   998 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 11.
I20260709 13:15:25.555887   894 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.557554  6967 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:25.558647  6967 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
I20260709 13:15:25.559779  6967 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.564544  7012 log.cc:927] Time spent T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:15:25.569612   999 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:25.573231  6947 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:25.586237   895 consensus_peers.cc:597] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.586819   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.591959   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.579532   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:15:25.599277  6885 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.600005  6918 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.599960  6885 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.600477  6996 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.601090  6996 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.601069   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.602761  6885 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.600510  6918 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.605201  6918 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.605979  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:25.592882   930 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:25.606802  1003 raft_consensus.cc:2393] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 12: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.610771   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.611828  7013 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.612411  7013 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.617211  6996 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.618367  6920 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.618417   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.619218   927 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 19.
I20260709 13:15:25.611400   930 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 12.
I20260709 13:15:25.619051  6920 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.621316  6920 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.621378  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:25.622282  1003 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 19: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.622054  1564 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.622893  6996 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:25.623293  6996 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.623966  6996 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.624696  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.625357  1595 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:15:25.626521   968 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:25.627271  6919 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.627663  6919 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.628031  6919 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.629675   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:15:25.640973  6920 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.641458  6920 raft_consensus.cc:493] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.641865  6920 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.633771  1564 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:15:25.646858   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.647372   930 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:25.636865  6919 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.655689  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.648867  6868 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:25.657039  6868 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:25.657423  6868 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:25.657742  6918 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.660640   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.665804  7013 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.674284  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.649350  6996 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.686658  1596 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 19.
I20260709 13:15:25.687814  6996 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.691524   751 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:25.713130   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 67873.6us
I20260709 13:15:25.672111  6919 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.715215   930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.715946   930 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.717042   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.721076   926 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:25.723948   926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.724514   926 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.724545   998 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:25.725894  7021 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.726011   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.726482  7021 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.720582   894 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:25.729235   998 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 19: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.732275   926 raft_consensus.cc:2468] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:25.735203  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4613bf5fb0446a80ac401fe0560967" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.718698  6918 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.734944  6885 consensus_queue.cc:1059] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 13:15:25.760622   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.777716   930 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:25.784602   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:25.785305   998 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 19.
I20260709 13:15:25.786346  6868 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.787191  7021 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.788552  6868 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:25.789170  1595 raft_consensus.cc:3060] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.798646  6920 raft_consensus.cc:515] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.816188  6920 leader_election.cc:290] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.818533  1595 raft_consensus.cc:2468] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:25.833830   968 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:25.834802  7022 raft_consensus.cc:2804] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:25.836915  7022 raft_consensus.cc:697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
W20260709 13:15:25.831880   969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111) [suppressed 321 similar messages]
I20260709 13:15:25.837863  7022 consensus_queue.cc:237] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.840929   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.843711  1593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.848945   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
W20260709 13:15:25.850257   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.856411  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:25.857095  1595 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 19.
I20260709 13:15:25.831882   927 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 19.
I20260709 13:15:25.857952  7017 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:25.857980  6863 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:25.859506   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:25.860689  6920 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:25.861187  6920 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.861608  6920 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.868022   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45ee7bf01de4a3ebb2eb56dcd597d5c" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.868867   927 raft_consensus.cc:2368] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13.
I20260709 13:15:25.873157  1564 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070
I20260709 13:15:25.874075  7023 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:25.874728  7023 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:25.876056  1595 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:25.875787  7023 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:25.879778  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:25.881037  1564 leader_election.cc:400] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13.
I20260709 13:15:25.881372  1564 leader_election.cc:403] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:15:25.882447  6863 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547) with higher term. Message: Invalid argument: T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13.
I20260709 13:15:25.881553   967 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:15:25.883944  6920 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.887455  7026 raft_consensus.cc:2804] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:25.888080  7026 raft_consensus.cc:697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:25.889031  7026 consensus_queue.cc:237] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.890125  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.892179  6996 raft_consensus.cc:3055] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:25.892591  6996 raft_consensus.cc:740] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:25.893443  6996 consensus_queue.cc:260] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.894582  6996 raft_consensus.cc:3060] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.894724   927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:25.895399   927 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 19.
I20260709 13:15:25.902063   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10b0600fc3f340eb999b12ed484fcdcb" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.902313   929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:25.902648   928 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.902832  7017 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.903321  7017 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.905565   930 raft_consensus.cc:1240] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:15:25.908388  6920 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.908421  6856 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:25.911166   928 raft_consensus.cc:2468] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 13.
I20260709 13:15:25.913326   930 raft_consensus.cc:1240] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:25.926893   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 18 to 19, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 19 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.928201   967 leader_election.cc:304] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:15:25.930089  6920 raft_consensus.cc:2804] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:25.932583  6920 raft_consensus.cc:697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:25.933629  6920 consensus_queue.cc:237] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.946138  6885 consensus_queue.cc:1059] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:15:25.946905   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:25.947973  6885 raft_consensus.cc:3055] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:25.948387  6885 raft_consensus.cc:740] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:25.950230  6885 consensus_queue.cc:260] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.961169  7017 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:25.963406  6885 raft_consensus.cc:3060] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:25.975066 32224 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 876)
I20260709 13:15:25.977833   751 catalog_manager.cc:5697] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 13, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.982297   929 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3bb5726887244caca146e176cc6bd070 in current term 20: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:25.984153   967 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:25.985241  7020 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.985950  7017 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 20 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:25.986554   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:25.988327   998 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 20: Already voted for candidate 3bb5726887244caca146e176cc6bd070 in this term.
I20260709 13:15:25.991433  1595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:25.992733   894 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ff541db50a49efa03669dfd777fc8e; no voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240
I20260709 13:15:25.994258  6856 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:15:26.013757  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:26.051741   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:26.058799   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:26.060611   751 catalog_manager.cc:5697] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:26.062512   751 catalog_manager.cc:5697] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 12 to 13, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 13 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:26.092945   999 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:26.095557  6967 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:26.109750  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:26.119829  1598 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:26.124078  7017 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:26.165803  1566 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:26.297372   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:26.308966   999 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:26.311477   929 raft_consensus.cc:1240] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Rejecting Update request from peer 597687e464174d1ea6043161e037d240 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:26.312542  7023 consensus_queue.cc:1059] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:26.313421  7023 raft_consensus.cc:3055] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:26.313728  7023 raft_consensus.cc:740] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:26.314543  7023 consensus_queue.cc:260] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:26.319991  7023 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:26.331976   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:26.370414  1598 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:15:26.370822  1598 pending_rounds.cc:85] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 9
I20260709 13:15:26.371058  1598 pending_rounds.cc:107] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:15:26.371388  1598 raft_consensus.cc:2889] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.374720  6920 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:15:26.383697  1566 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:26.421535  1595 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:15:26.424041  7026 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:26.426416  1598 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:15:26.426793  1598 pending_rounds.cc:85] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 9
I20260709 13:15:26.427003  1598 pending_rounds.cc:107] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:15:26.427214  1598 raft_consensus.cc:2889] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.432730   929 raft_consensus.cc:1275] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:26.434002   928 raft_consensus.cc:1275] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:26.435680  6919 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:26.438009   930 raft_consensus.cc:1275] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:15:26.472733  1566 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:26.494673  6920 consensus_queue.cc:1048] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:26.496080  6920 consensus_queue.cc:1048] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:26.501797  7026 consensus_queue.cc:1048] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:15:26.606774  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:26.638952  7027 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.639660  7027 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:26.642530  7027 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:26.659775  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:26.684828   894 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.120.62:45937, user_credentials={real_user=slave}} blocked reactor thread for 49741.4us
I20260709 13:15:26.712870   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:26.713569   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:15:26.737293  1564 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:26.738330  7028 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:26.738760  7028 raft_consensus.cc:493] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.739109  7028 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:26.750264  7028 raft_consensus.cc:515] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:26.752607  7028 leader_election.cc:290] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Requested vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:26.778508  1566 leader_election.cc:336] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:26.782887   894 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.31.120.62:45937, user_credentials={real_user=slave}} blocked reactor thread for 64455.7us
I20260709 13:15:26.809861   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec15122641447a3bd48017eb990578d" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:26.810552   928 raft_consensus.cc:3060] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:26.803828   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 75070.2us
W20260709 13:15:26.828066  1566 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:26.871821   928 raft_consensus.cc:2468] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 17.
W20260709 13:15:26.878738   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:26.884269   895 consensus_peers.cc:597] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:26.890641  1564 leader_election.cc:304] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:26.906956  7023 raft_consensus.cc:2804] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:26.907640  7023 raft_consensus.cc:697] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:26.908722  7023 consensus_queue.cc:237] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:26.936906   969 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:26.966066   751 catalog_manager.cc:5697] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 14 to 17, leader changed from 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 17 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:26.975620 32224 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 876)
W20260709 13:15:27.072093  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:27.137943  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:27.344694   928 raft_consensus.cc:1275] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:27.370231  7023 consensus_queue.cc:1048] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:15:27.389078   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:27.419325   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:27.434110  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:27.474804  1566 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:27.479725  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:27.533775   896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 181950us
I20260709 13:15:27.973289  7054 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.976737  7054 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:27.981285  7054 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:27.989563 32224 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 876)
I20260709 13:15:28.009199  7023 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.009805  7023 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:28.012305  7023 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:28.082072   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.083024   895 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:28.083595   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.085729   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.223664  1003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:28.224628  1003 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 20.
W20260709 13:15:28.245150   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.249626   895 consensus_peers.cc:597] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.256314   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.266398  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:28.267298  1597 raft_consensus.cc:2410] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e for term 21 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 13:15:28.271919   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e" is_pre_election: true
I20260709 13:15:28.272608   928 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 20.
W20260709 13:15:28.301254  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 80535.1us
I20260709 13:15:28.304965   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:28.305739   999 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 20.
I20260709 13:15:28.310681  1564 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e; no voters: 
I20260709 13:15:28.312526  7065 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:28.312992  7065 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.313337  7065 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:28.325977   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.337224   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 597687e464174d1ea6043161e037d240
I20260709 13:15:28.347016  7054 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:28.347508  7054 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.348898  7054 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:28.344465  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:28.356820  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:28.356225   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.358258   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.378438  7054 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:28.384847   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:28.385598   999 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:28.380367  7065 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:28.398062   999 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ff541db50a49efa03669dfd777fc8e in term 21.
I20260709 13:15:28.400240   928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "91ff541db50a49efa03669dfd777fc8e"
I20260709 13:15:28.404130   928 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 21: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:28.404543   896 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e; no voters: 
W20260709 13:15:28.382097   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.405647  7048 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:15:28.405980   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:28.406288  7048 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 LEADER]: Becoming Leader. State: Replica: 91ff541db50a49efa03669dfd777fc8e, State: Running, Role: LEADER
W20260709 13:15:28.406502   969 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:28.401654  7065 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:28.409945   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:28.410872   998 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 597687e464174d1ea6043161e037d240 in current term 21: Already voted for candidate 91ff541db50a49efa03669dfd777fc8e in this term.
I20260709 13:15:28.407302  7048 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:28.415412  1567 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:28.418447  7065 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:28.421064  7054 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [CANDIDATE]: Term 21 election: Requested vote from peers 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:28.421470  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "91ff541db50a49efa03669dfd777fc8e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:28.422267  1597 raft_consensus.cc:2393] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ff541db50a49efa03669dfd777fc8e in current term 21: Already voted for candidate 597687e464174d1ea6043161e037d240 in this term.
I20260709 13:15:28.457607   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e reported cstate change: term changed from 19 to 21, leader changed from 597687e464174d1ea6043161e037d240 (127.31.120.4) to 91ff541db50a49efa03669dfd777fc8e (127.31.120.2). New cstate: current_term: 21 leader_uuid: "91ff541db50a49efa03669dfd777fc8e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:28.577150  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.636726   951 debug-util.cc:398] Leaking SignalData structure 0x7b080030ec40 after lost signal to thread 32227
W20260709 13:15:28.637987   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003848a0 after lost signal to thread 787
W20260709 13:15:28.639303   951 debug-util.cc:398] Leaking SignalData structure 0x7b0800298f80 after lost signal to thread 954
W20260709 13:15:28.640553   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2840 after lost signal to thread 1028
W20260709 13:15:28.641891   951 debug-util.cc:398] Leaking SignalData structure 0x7b08003ecc00 after lost signal to thread 1625
W20260709 13:15:28.691556   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.700588   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.701367   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.767810  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:28.769358  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.784852   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.801653   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.810297  1566 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.812709   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:28.813796   998 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:28.815428  6967 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:28.823593  1597 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Refusing update from remote peer 91ff541db50a49efa03669dfd777fc8e: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:15:28.824036  1597 pending_rounds.cc:85] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Aborting all ops after (but not including) 16
I20260709 13:15:28.824245  1597 pending_rounds.cc:107] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: Aborting uncommitted NO_OP operation due to leader change: 19.17
I20260709 13:15:28.824468  1597 raft_consensus.cc:2889] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:28.827024  6967 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:15:28.903470   969 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.937804   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:28.940760   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:28.990079 32224 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 876)
W20260709 13:15:29.166397  1566 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.178269   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.267572   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.272503  1566 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.292799  1566 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.334877   895 consensus_peers.cc:597] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.352977   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.752031  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.782106  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.795773  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.819904   969 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:29.842465  1566 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.898319   895 consensus_peers.cc:597] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:29.967844   895 consensus_peers.cc:597] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:29.990619 32224 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 876)
W20260709 13:15:30.164565  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:30.204037   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.253993   969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 50015.1us
W20260709 13:15:30.257015  1566 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:30.316881  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:30.341899  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:30.357316   895 consensus_peers.cc:597] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.419459   895 consensus_peers.cc:597] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.476679   895 consensus_peers.cc:597] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.619006   969 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:30.677747   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.679709   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.684648  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:30.698726   895 consensus_peers.cc:597] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.793751   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.796029   895 consensus_peers.cc:597] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:30.817965   895 consensus_peers.cc:597] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:30.832870  1566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111) [suppressed 411 similar messages]
W20260709 13:15:30.908598   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:30.980605 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
W20260709 13:15:31.006582  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.012228   895 consensus_peers.cc:597] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:31.015208   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:31.035272  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.058449   894 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 62991.4us
W20260709 13:15:31.129181  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:31.132510 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:15:31.138300  1564 consensus_peers.cc:597] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.175747   895 consensus_peers.cc:597] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:31.198015   967 consensus_peers.cc:597] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.206660  1564 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.228248   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:31.240772   967 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.251345   967 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.284302   967 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.289222  1566 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.311954  1564 consensus_peers.cc:597] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.314946   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:31.321004   967 consensus_peers.cc:597] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.322158   967 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.331908  1564 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.327545   967 consensus_peers.cc:597] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.344307   967 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.348228   967 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.351300  1564 consensus_peers.cc:597] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.367055   967 consensus_peers.cc:597] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.371030   967 consensus_peers.cc:597] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.386559 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:31.387966 32224 tablet_replica.cc:333] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.388888 32224 raft_consensus.cc:2243] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.389518 32224 raft_consensus.cc:2272] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.394232   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:31.394300 32224 tablet_replica.cc:333] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
W20260709 13:15:31.395123   967 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.395418 32224 raft_consensus.cc:2243] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.396644 32224 raft_consensus.cc:2272] T 82ed8549f06d4ef3a4620f596e0139b7 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.399197 32224 tablet_replica.cc:333] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.400069 32224 raft_consensus.cc:2243] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.400682 32224 raft_consensus.cc:2272] T 1b4d7e87915947148833db3fe84cf623 P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.403496 32224 tablet_replica.cc:333] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.417778 32224 raft_consensus.cc:2243] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.418496 32224 raft_consensus.cc:2272] T 56cfb3f7cad745918e1bc1faf5cc697c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.422863   969 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.442626  1564 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.447494   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:31.448740 32224 tablet_replica.cc:333] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.449790 32224 raft_consensus.cc:2243] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.450454 32224 raft_consensus.cc:2272] T 14a8781662d54452a1478c2ae0e3f911 P 91ff541db50a49efa03669dfd777fc8e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.454376 32224 tablet_replica.cc:333] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
W20260709 13:15:31.461735  1564 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.470683 32224 raft_consensus.cc:2243] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.472496 32224 raft_consensus.cc:2272] T 24f44707b1d3466e91403f063187a9b2 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.475219 32224 tablet_replica.cc:333] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.476081 32224 raft_consensus.cc:2243] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.481158 32224 raft_consensus.cc:2272] T 248dca75dd144af2a2106e703fe92de4 P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.484061 32224 tablet_replica.cc:333] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.484932 32224 raft_consensus.cc:2243] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.486204 32224 raft_consensus.cc:2272] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.497367   967 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.504271  1564 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.512797 32224 tablet_replica.cc:333] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.513907 32224 raft_consensus.cc:2243] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:31.515674  1564 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.514752 32224 raft_consensus.cc:2272] T 2ec15122641447a3bd48017eb990578d P 91ff541db50a49efa03669dfd777fc8e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.520097  1564 consensus_peers.cc:597] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.521854 32224 tablet_replica.cc:333] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.522869 32224 raft_consensus.cc:2243] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.523504 32224 raft_consensus.cc:2272] T cdfea09a06ff4100a0b80619f35f7008 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.526459 32224 tablet_replica.cc:333] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
W20260709 13:15:31.530433   967 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.538609 32224 raft_consensus.cc:2243] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.539475 32224 raft_consensus.cc:2272] T 2b38ec4f420f4de9883306887a006c24 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.540334  1564 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.542550 32224 tablet_replica.cc:333] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.543562 32224 raft_consensus.cc:2243] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.544214 32224 raft_consensus.cc:2272] T 2b4613bf5fb0446a80ac401fe0560967 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.547250 32224 tablet_replica.cc:333] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
W20260709 13:15:31.554432  1564 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.564282 32224 raft_consensus.cc:2243] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.565280 32224 raft_consensus.cc:2272] T f379c9f3a933409ca08bceb325f43132 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.586815   967 consensus_peers.cc:597] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.607151   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:31.611501   967 consensus_peers.cc:597] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:31.627411  1566 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.650575   895 consensus_peers.cc:597] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:31.655153 32224 tablet_replica.cc:333] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.658960 32224 raft_consensus.cc:2243] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.659644 32224 raft_consensus.cc:2272] T 5ed094558dad4fb2b45bc747bbc6adba P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.662362  1566 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:31.662871 32224 tablet_replica.cc:333] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.669034 32224 raft_consensus.cc:2243] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.670266 32224 raft_consensus.cc:2272] T 633fa6f0c6434c49aeb87667c9d97141 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.673013 32224 tablet_replica.cc:333] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.676236 32224 raft_consensus.cc:2243] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.676838 32224 raft_consensus.cc:2272] T d3d5a097d0fe49bab1a3ec1f78f4164d P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.686515 32224 tablet_replica.cc:333] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.687893 32224 raft_consensus.cc:2243] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.688632 32224 raft_consensus.cc:2272] T c43ab43025004edcacdb3d8a30106319 P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.691299 32224 tablet_replica.cc:333] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.704339 32224 raft_consensus.cc:2243] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Raft consensus shutting down.
W20260709 13:15:31.695231  1564 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:31.705554 32224 raft_consensus.cc:2272] T b8c347b6586f486192f48c46974a2526 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.715022 32224 tablet_replica.cc:333] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.716079 32224 raft_consensus.cc:2243] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.717305 32224 raft_consensus.cc:2272] T c72e2f13b3404583a1c34e15098c42b9 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.719944 32224 tablet_replica.cc:333] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.722381 32224 raft_consensus.cc:2243] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.723448 32224 raft_consensus.cc:2272] T a4018887e34a40108555fd3cbaa2c975 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.726179 32224 tablet_replica.cc:333] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.736150 32224 raft_consensus.cc:2243] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.771692 32224 raft_consensus.cc:2272] T 731bda621b364aaa81d868d0cef5e739 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.774689 32224 tablet_replica.cc:333] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.775523 32224 raft_consensus.cc:2243] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.776198 32224 raft_consensus.cc:2272] T 6874f81648724ea980572af4b6fa79d2 P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.796583 32224 tablet_replica.cc:333] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.797456 32224 raft_consensus.cc:2243] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.798609 32224 raft_consensus.cc:2272] T 9155790402004e4f930983841dfd8c64 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.819181 32224 tablet_replica.cc:333] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.824270 32224 raft_consensus.cc:2243] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.844606 32224 raft_consensus.cc:2272] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.847321 32224 tablet_replica.cc:333] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.856375 32224 raft_consensus.cc:2243] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.857585 32224 raft_consensus.cc:2272] T ef39166ba76a41d18b3770b743ea3488 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.872196 32224 tablet_replica.cc:333] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.873157 32224 raft_consensus.cc:2243] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:31.873456   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:31.873819 32224 raft_consensus.cc:2272] T c3247a08a45747539334328f4abf2acc P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.876430 32224 tablet_replica.cc:333] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.877295 32224 raft_consensus.cc:2243] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.877905 32224 raft_consensus.cc:2272] T 9794c08fc6b2428b981acd6c3c238b5f P 91ff541db50a49efa03669dfd777fc8e [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.880517 32224 tablet_replica.cc:333] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.881378 32224 raft_consensus.cc:2243] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.882041 32224 raft_consensus.cc:2272] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.884800 32224 tablet_replica.cc:333] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.889449 32224 raft_consensus.cc:2243] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.890534 32224 raft_consensus.cc:2272] T 35b46c75a1e049dd910275709bd62839 P 91ff541db50a49efa03669dfd777fc8e [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.893029 32224 tablet_replica.cc:333] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.903656 32224 raft_consensus.cc:2243] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.905596 32224 raft_consensus.cc:2272] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.908334 32224 tablet_replica.cc:333] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.916951 32224 raft_consensus.cc:2243] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.918170 32224 raft_consensus.cc:2272] T 31088ed18ff54bcea05d848f8c67e437 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.923101 32224 tablet_replica.cc:333] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.927683 32224 raft_consensus.cc:2243] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.928725 32224 raft_consensus.cc:2272] T 3b871863269841f7b0aa8baf5e640865 P 91ff541db50a49efa03669dfd777fc8e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.930261  1566 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:31.932554 32224 tablet_replica.cc:333] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.939478 32224 raft_consensus.cc:2243] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.940510 32224 raft_consensus.cc:2272] T 01fc410be7f74515adce264a85731b90 P 91ff541db50a49efa03669dfd777fc8e [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.943243 32224 tablet_replica.cc:333] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.948205 32224 raft_consensus.cc:2243] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.949364 32224 raft_consensus.cc:2272] T 9ee81d93e84c4c689b02deb555210ba4 P 91ff541db50a49efa03669dfd777fc8e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.953011 32224 tablet_replica.cc:333] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:31.960222 32224 raft_consensus.cc:2243] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.960912 32224 raft_consensus.cc:2272] T e65c03394bf04e2fb03562b27d85f5b3 P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:32.011461   969 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:32.017540   895 consensus_peers.cc:597] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:32.036716  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:32.097707 32224 tablet_replica.cc:333] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.098407 32224 raft_consensus.cc:2243] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:32.099308 32224 raft_consensus.cc:2272] T 6d9b9f004a6f4293a658cca1bb92e319 P 91ff541db50a49efa03669dfd777fc8e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.102252 32224 tablet_replica.cc:333] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.103132 32224 raft_consensus.cc:2243] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.104839 32224 raft_consensus.cc:2272] T 4997f2c7b33048a5b010a4e0c7866fa2 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.107386 32224 tablet_replica.cc:333] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.108213 32224 raft_consensus.cc:2243] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.108807 32224 raft_consensus.cc:2272] T 7de58b0d5ad94d869fac3b0491143543 P 91ff541db50a49efa03669dfd777fc8e [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.110785 32224 tablet_replica.cc:333] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.111348 32224 raft_consensus.cc:2243] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.111872 32224 raft_consensus.cc:2272] T 10b0600fc3f340eb999b12ed484fcdcb P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.113850 32224 tablet_replica.cc:333] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.114382 32224 raft_consensus.cc:2243] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.114856 32224 raft_consensus.cc:2272] T bdca38d5137a4409b2e61eaf2e13fd7b P 91ff541db50a49efa03669dfd777fc8e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.117424 32224 tablet_replica.cc:333] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.118072 32224 raft_consensus.cc:2243] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:32.119259 32224 raft_consensus.cc:2272] T cfd817716cdc4448b3f884eecc94351c P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.121814 32224 tablet_replica.cc:333] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.122507 32224 raft_consensus.cc:2243] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.123463 32224 raft_consensus.cc:2272] T bedf1cefa52d44d884e14c5261c6437a P 91ff541db50a49efa03669dfd777fc8e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.126531 32224 tablet_replica.cc:333] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.127214 32224 raft_consensus.cc:2243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.127805 32224 raft_consensus.cc:2272] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 91ff541db50a49efa03669dfd777fc8e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.130659 32224 tablet_replica.cc:333] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.131376 32224 raft_consensus.cc:2243] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.132332 32224 raft_consensus.cc:2272] T 701b27559f8e47f9b63e5d220d3d5a58 P 91ff541db50a49efa03669dfd777fc8e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:32.134837 32224 tablet_replica.cc:333] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e: stopping tablet replica
I20260709 13:15:32.135514 32224 raft_consensus.cc:2243] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:32.136122 32224 raft_consensus.cc:2272] T 6cfaab49f56546a49978c6055f57640f P 91ff541db50a49efa03669dfd777fc8e [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:32.384845  1566 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:32.389796  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:32.406828   968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:33689, user_credentials={real_user=slave}} blocked reactor thread for 74522.1us
W20260709 13:15:32.453136  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:32.481683  1567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:44411, user_credentials={real_user=slave}} blocked reactor thread for 123798us
W20260709 13:15:32.494218  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:32.499843  7101 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.500468  7101 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.502612  7101 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:32.507786   967 leader_election.cc:336] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:32.515651  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
W20260709 13:15:32.523136   969 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:32.528924   968 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.530138  7103 raft_consensus.cc:2749] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:32.562517  7105 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.563210  7105 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.567226   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:32.567884   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:15:32.569123  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:32.570086  7115 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:32.570526  7115 raft_consensus.cc:493] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.570870  7115 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:32.574314  7105 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:32.574941  1564 leader_election.cc:336] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.579099  7115 raft_consensus.cc:515] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.583201   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ed8549f06d4ef3a4620f596e0139b7" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:32.583832   998 raft_consensus.cc:3060] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:32.584955  7115 leader_election.cc:290] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:32.586601  1564 leader_election.cc:336] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.590889   998 raft_consensus.cc:2468] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 17.
I20260709 13:15:32.593726  1567 leader_election.cc:304] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.594913  7105 raft_consensus.cc:2804] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:32.595441  7105 raft_consensus.cc:697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:32.596380  7105 consensus_queue.cc:237] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.621482   751 catalog_manager.cc:5697] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 16 to 17, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 17 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:32.696331  7101 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.696949  7101 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:32.701090  7103 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.705430  7103 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.712653  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:32.714413  7103 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:32.716099  7101 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:32.716679   967 leader_election.cc:336] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.717612   967 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 597687e464174d1ea6043161e037d240, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.719695  7101 raft_consensus.cc:2749] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:32.726873   967 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:32.733644   969 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:32.734197   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.735112  7101 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:32.761858   969 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:32.803972  7101 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.808224  7101 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
W20260709 13:15:32.822559  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:32.831342  7101 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:32.858605   969 leader_election.cc:336] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:32.867796   967 leader_election.cc:336] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.868510   967 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.869367  7103 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:32.906729  7103 raft_consensus.cc:493] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.907246  7103 raft_consensus.cc:515] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:32.909669  7103 leader_election.cc:290] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:32.914239  1566 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:32.914436   969 leader_election.cc:336] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:32.920658   967 leader_election.cc:336] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.921075   967 leader_election.cc:304] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.921689  7103 raft_consensus.cc:2749] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:32.933708  7105 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.934353  7105 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.936379  7105 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:32.937057  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:32.942788  7105 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
W20260709 13:15:32.942970  1566 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:32.944087  7105 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.947245  7105 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:32.956740  1564 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.957197  1564 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.957924  7105 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:32.958636  1566 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:32.959018   999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:32.960521  1567 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:32.961673  7115 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:32.968340  1564 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.971912  7100 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.972448  7100 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.974491  7100 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:32.976244   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:32.978000  7100 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:32.978623  7100 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:32.982031  7100 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:32.983150  1564 leader_election.cc:336] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:32.983675  1564 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:32.984566  7100 raft_consensus.cc:2749] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:32.991951  1566 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:32.998890   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:33.000470  1567 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:33.000907  7103 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.001439  7103 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:33.002631  7105 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:33.005352  7103 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:33.009871   967 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.010641  1564 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.015852   969 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.016337   969 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.017026  7103 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:33.038636  7105 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.039181  7105 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.041361  7105 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:33.048266  1564 consensus_peers.cc:597] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:33.048724  1564 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.049822  1566 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.050213  1566 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.050889  7105 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:33.055238  7105 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.055765  7105 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.057865  7105 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:33.059022   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
W20260709 13:15:33.064522  1564 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.064997  1564 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.066047  7105 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:33.070286  7107 raft_consensus.cc:493] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.070926  7107 raft_consensus.cc:515] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.077270  7107 leader_election.cc:290] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:33.077714   967 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.078265  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9d8e1d04724f1ba621ecf90cc49b82" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:33.078912  1597 raft_consensus.cc:2468] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
W20260709 13:15:33.093240   969 leader_election.cc:336] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.095422   969 leader_election.cc:304] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.097344  7107 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.097980  7107 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:33.101243  7103 raft_consensus.cc:2749] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:33.103093   969 consensus_peers.cc:597] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:33.104841  7107 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:33.106989   967 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.110533   969 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.111006   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.112159  7107 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:33.113826   998 raft_consensus.cc:1275] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:33.115511  7105 consensus_queue.cc:1048] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:33.130290   969 consensus_peers.cc:597] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.146610  7105 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.147270  7105 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:33.158696  1564 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.159809  7105 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:33.163053  1566 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.163527  1566 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.164820  7115 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:33.164868  7105 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.165931  7105 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.167740  7105 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:33.172528  1564 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.173565  1566 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.173923  1566 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.174510  7105 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:33.184182  1622 debug-util.cc:398] Leaking SignalData structure 0x7b0800171a00 after lost signal to thread 32227
W20260709 13:15:33.185164  1622 debug-util.cc:398] Leaking SignalData structure 0x7b080009c0a0 after lost signal to thread 787
W20260709 13:15:33.185663  1622 debug-util.cc:398] Leaking SignalData structure 0x7b080031c680 after lost signal to thread 954
W20260709 13:15:33.186678  1622 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1b80 after lost signal to thread 1028
W20260709 13:15:33.188174  1622 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1280 after lost signal to thread 1625
I20260709 13:15:33.225663  7105 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.226454  7105 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.228984  7105 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:33.230486   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070" is_pre_election: true
I20260709 13:15:33.231163   998 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 15.
I20260709 13:15:33.235953  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
W20260709 13:15:33.236837  1564 leader_election.cc:336] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.237861  7105 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:33.238426  7105 raft_consensus.cc:493] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.238816  7105 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:33.242966  7103 raft_consensus.cc:493] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.243530  7103 raft_consensus.cc:515] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:33.245721  7103 leader_election.cc:290] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:33.247279  7105 raft_consensus.cc:515] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:33.255959   967 leader_election.cc:336] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.256946   969 leader_election.cc:336] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:33.257366   969 leader_election.cc:304] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:33.258122  1564 leader_election.cc:336] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.259513  7105 leader_election.cc:290] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:33.264559  7103 raft_consensus.cc:2749] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:33.275396   998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c347b6586f486192f48c46974a2526" candidate_uuid: "597687e464174d1ea6043161e037d240" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3bb5726887244caca146e176cc6bd070"
I20260709 13:15:33.276121   998 raft_consensus.cc:3060] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:33.285785  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:33.295657  7107 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.297708  7107 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.304241  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:33.304947  1596 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 21.
I20260709 13:15:33.306619   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:33.308696  7101 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:33.309234  7101 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.309636  7101 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:33.320518  7103 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.321090  7103 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.323096  7103 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
W20260709 13:15:33.330027   967 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.291249   998 raft_consensus.cc:2468] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597687e464174d1ea6043161e037d240 in term 16.
I20260709 13:15:33.339564  1596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:33.331179  7107 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:33.340422  1596 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 11.
I20260709 13:15:33.351294  7101 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.354324  7101 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:33.357599  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee81d93e84c4c689b02deb555210ba4" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:33.358031   968 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:33.358426  1597 raft_consensus.cc:3060] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:33.359849   969 consensus_peers.cc:597] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.360735  1567 leader_election.cc:304] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
W20260709 13:15:33.362391   967 leader_election.cc:336] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.362026  7115 raft_consensus.cc:2804] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:33.361228  7130 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:33.368605  7130 raft_consensus.cc:493] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.368692  7115 raft_consensus.cc:697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 16 LEADER]: Becoming Leader. State: Replica: 597687e464174d1ea6043161e037d240, State: Running, Role: LEADER
I20260709 13:15:33.369089  7130 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:33.369709  7115 consensus_queue.cc:237] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.371868  1597 raft_consensus.cc:2468] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 22.
W20260709 13:15:33.374886   967 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:33.376222   968 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.381363  7101 raft_consensus.cc:2804] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:33.381877  7101 raft_consensus.cc:697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 22 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:33.384768  7101 consensus_queue.cc:237] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.400835   751 catalog_manager.cc:5697] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 reported cstate change: term changed from 15 to 16, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 597687e464174d1ea6043161e037d240 (127.31.120.4). New cstate: current_term: 16 leader_uuid: "597687e464174d1ea6043161e037d240" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:33.403110   969 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.403402  7128 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:33.403955  7128 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.406191  7128 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:33.407231  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb0c412ac6a42fe84d4b2ce85773e26" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240" is_pre_election: true
I20260709 13:15:33.407932  1597 raft_consensus.cc:2468] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 16.
W20260709 13:15:33.411441   967 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.430464   969 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:33.433396  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:33.435200   969 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:33.435001  7130 raft_consensus.cc:515] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:33.440816  7129 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:33.444329  7130 leader_election.cc:290] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Requested vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 597687e464174d1ea6043161e037d240 (127.31.120.4:33689)
I20260709 13:15:33.445547  1597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35b46c75a1e049dd910275709bd62839" candidate_uuid: "3bb5726887244caca146e176cc6bd070" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "597687e464174d1ea6043161e037d240"
I20260709 13:15:33.446228  1597 raft_consensus.cc:3060] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:33.457508   752 catalog_manager.cc:5697] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 21 to 22, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 22 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:33.458395  1597 raft_consensus.cc:2468] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bb5726887244caca146e176cc6bd070 in term 12.
I20260709 13:15:33.463483   968 leader_election.cc:304] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bb5726887244caca146e176cc6bd070, 597687e464174d1ea6043161e037d240; no voters: 
I20260709 13:15:33.464288  7130 raft_consensus.cc:2804] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:33.464751  7130 raft_consensus.cc:697] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Becoming Leader. State: Replica: 3bb5726887244caca146e176cc6bd070, State: Running, Role: LEADER
I20260709 13:15:33.465595  7130 consensus_queue.cc:237] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:33.489091   967 leader_election.cc:336] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:33.498384   969 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.502130   752 catalog_manager.cc:5697] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 reported cstate change: term changed from 11 to 12, leader changed from 91ff541db50a49efa03669dfd777fc8e (127.31.120.2) to 3bb5726887244caca146e176cc6bd070 (127.31.120.3). New cstate: current_term: 12 leader_uuid: "3bb5726887244caca146e176cc6bd070" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:33.531625   969 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:33.612972  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:33.616437   967 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.673069  1564 consensus_peers.cc:597] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.711290   967 consensus_peers.cc:597] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.746977   967 consensus_peers.cc:597] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.780248   969 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:33.823040   967 consensus_peers.cc:597] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:33.880877  1564 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:33.900615  1596 raft_consensus.cc:1275] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:15:33.902318  7101 consensus_queue.cc:1048] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:15:33.935539  1596 raft_consensus.cc:1275] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Refusing update from remote peer 3bb5726887244caca146e176cc6bd070: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:15:33.954663  1566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:43377, user_credentials={real_user=slave}} blocked reactor thread for 58445.4us
I20260709 13:15:33.960623  7126 consensus_queue.cc:1048] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:15:33.963512   967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:34547, user_credentials={real_user=slave}} blocked reactor thread for 52921.6us
W20260709 13:15:33.967998   969 consensus_peers.cc:597] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.979799   999 raft_consensus.cc:1275] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Refusing update from remote peer 597687e464174d1ea6043161e037d240: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:15:33.987969  1566 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:33.990828   969 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:33.992437  7115 consensus_queue.cc:1048] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:15:33.995138   967 consensus_peers.cc:597] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.996397   967 consensus_peers.cc:597] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.000125  1564 consensus_peers.cc:597] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.002527  1564 consensus_peers.cc:597] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.010689   967 consensus_peers.cc:597] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.012534  1566 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.055298   967 consensus_peers.cc:597] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.056823   967 consensus_peers.cc:597] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.057595   967 consensus_peers.cc:597] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.058811  1564 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.088246   967 consensus_peers.cc:597] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.089136   967 consensus_peers.cc:597] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.101784  1564 consensus_peers.cc:597] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.102540  1564 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.106243   967 consensus_peers.cc:597] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.106950   967 consensus_peers.cc:597] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.109825  1564 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.114017   967 consensus_peers.cc:597] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.110844   969 consensus_peers.cc:597] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:34.128314  1564 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.148424   969 consensus_peers.cc:597] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.179098  1564 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.188411  1564 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.207788   967 consensus_peers.cc:597] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.254926  1566 consensus_peers.cc:597] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.275174   969 consensus_peers.cc:597] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:34.276294  1564 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.295079   967 consensus_peers.cc:597] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:34.303366 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
W20260709 13:15:34.334524  1564 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.355784   967 consensus_peers.cc:597] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.416540  1566 consensus_peers.cc:597] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.420265  1566 consensus_peers.cc:597] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.452910  1564 consensus_peers.cc:597] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:34.487505   969 consensus_peers.cc:597] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:34.525204  1566 consensus_peers.cc:597] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:34.569026  7126 raft_consensus.cc:493] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.571457  7126 raft_consensus.cc:515] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:34.575086  7126 leader_election.cc:290] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:34.582136   969 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:34.590088   967 leader_election.cc:336] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:34.590699   967 leader_election.cc:304] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.591667 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:15:34.591795  7126 raft_consensus.cc:2749] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:34.614523  1567 consensus_peers.cc:597] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.626434  1567 consensus_peers.cc:597] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.630833  1567 consensus_peers.cc:597] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.656606  1567 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:34.684283 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:34.689273 32224 tablet_replica.cc:333] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.692584 32224 raft_consensus.cc:2243] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.694056 32224 raft_consensus.cc:2272] T 56cfb3f7cad745918e1bc1faf5cc697c P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.697695 32224 tablet_replica.cc:333] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.698572  7126 raft_consensus.cc:493] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.699147  7126 raft_consensus.cc:515] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:34.700227 32224 raft_consensus.cc:2243] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.701493  7126 leader_election.cc:290] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:34.701689 32224 raft_consensus.cc:2272] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.704869 32224 tablet_replica.cc:333] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.705829 32224 raft_consensus.cc:2243] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.706496 32224 raft_consensus.cc:2272] T 82ed8549f06d4ef3a4620f596e0139b7 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.709331 32224 tablet_replica.cc:333] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.710194 32224 raft_consensus.cc:2243] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.712559 32224 raft_consensus.cc:2272] T 6533a74f2a524baab75bdfae9b5b1888 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.713352   967 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:34.714178   969 leader_election.cc:336] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:34.714622   969 leader_election.cc:304] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.715655 32224 tablet_replica.cc:333] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.715758  7126 raft_consensus.cc:2749] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:34.716980 32224 raft_consensus.cc:2243] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.719646 32224 raft_consensus.cc:2272] T dcb3d540c3fc49bf926540f7bd349110 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.722788 32224 tablet_replica.cc:333] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.723460  7126 raft_consensus.cc:493] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.724253 32224 raft_consensus.cc:2243] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.724174  7126 raft_consensus.cc:515] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:34.725852 32224 raft_consensus.cc:2272] T 1b4d7e87915947148833db3fe84cf623 P 3bb5726887244caca146e176cc6bd070 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.726653  7126 leader_election.cc:290] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:34.729151 32224 tablet_replica.cc:333] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.730132 32224 raft_consensus.cc:2243] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.736599 32224 raft_consensus.cc:2272] T 2840f4bbf21443318b9ad41dc7e3e853 P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.740413 32224 tablet_replica.cc:333] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.741319 32224 raft_consensus.cc:2243] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.742228 32224 raft_consensus.cc:2272] T e0cca622722b48a7ba611a3b77376f80 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.742798   967 leader_election.cc:336] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:34.744021   969 leader_election.cc:336] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:34.744529   969 leader_election.cc:304] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.745481 32224 tablet_replica.cc:333] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.745909  7126 raft_consensus.cc:2749] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:34.746457 32224 raft_consensus.cc:2243] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.747084 32224 raft_consensus.cc:2272] T 14a8781662d54452a1478c2ae0e3f911 P 3bb5726887244caca146e176cc6bd070 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.750137 32224 tablet_replica.cc:333] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.751252 32224 raft_consensus.cc:2243] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.752966 32224 raft_consensus.cc:2272] T 516d927af6bf46aa947c0919f1dfc830 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.755997 32224 tablet_replica.cc:333] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.756626 32224 raft_consensus.cc:2243] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.757714 32224 raft_consensus.cc:2272] T f379c9f3a933409ca08bceb325f43132 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.760532 32224 tablet_replica.cc:333] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.761561 32224 raft_consensus.cc:2243] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.762854 32224 raft_consensus.cc:2272] T 2b4613bf5fb0446a80ac401fe0560967 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.774634 32224 tablet_replica.cc:333] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.775974 32224 raft_consensus.cc:2243] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.777011 32224 raft_consensus.cc:2272] T 95c3b97a883a4aeb97bdcc43b82a2b95 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.781873 32224 tablet_replica.cc:333] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
W20260709 13:15:34.782481  1566 consensus_peers.cc:597] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:34.782757 32224 raft_consensus.cc:2243] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 LEADER]: Raft consensus shutting down.
W20260709 13:15:34.783159  1566 consensus_peers.cc:597] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:34.784565 32224 raft_consensus.cc:2272] T 5ed094558dad4fb2b45bc747bbc6adba P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.787729  1567 consensus_peers.cc:597] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:34.789039  1567 consensus_peers.cc:597] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:34.789744 32224 tablet_replica.cc:333] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.790622 32224 raft_consensus.cc:2243] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.791343 32224 raft_consensus.cc:2272] T 248dca75dd144af2a2106e703fe92de4 P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.794202 32224 tablet_replica.cc:333] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.795068 32224 raft_consensus.cc:2243] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.796612 32224 raft_consensus.cc:2272] T cdfea09a06ff4100a0b80619f35f7008 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.799274 32224 tablet_replica.cc:333] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.801460 32224 raft_consensus.cc:2243] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.804723 32224 raft_consensus.cc:2272] T 91555b4ec53349c287c5221b9030c23a P 3bb5726887244caca146e176cc6bd070 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.807657 32224 tablet_replica.cc:333] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.808570 32224 raft_consensus.cc:2243] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.809996 32224 raft_consensus.cc:2272] T 2b38ec4f420f4de9883306887a006c24 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.810089  1567 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:34.812645  7130 raft_consensus.cc:493] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.813166  7130 raft_consensus.cc:515] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } }
I20260709 13:15:34.813844 32224 tablet_replica.cc:333] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.814707 32224 raft_consensus.cc:2243] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.815438 32224 raft_consensus.cc:2272] T 53d4a96b7f094cf691b31dbe6fb015bd P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.815373  7130 leader_election.cc:290] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:34.818538 32224 tablet_replica.cc:333] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.819393 32224 raft_consensus.cc:2243] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.822118 32224 raft_consensus.cc:2272] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.823974   967 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:34.825284 32224 tablet_replica.cc:333] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.826236 32224 raft_consensus.cc:2243] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.827024 32224 raft_consensus.cc:2272] T d3d5a097d0fe49bab1a3ec1f78f4164d P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.830163 32224 tablet_replica.cc:333] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.831256 32224 raft_consensus.cc:2243] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.832001 32224 raft_consensus.cc:2272] T b8c347b6586f486192f48c46974a2526 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.835322 32224 tablet_replica.cc:333] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.836351 32224 raft_consensus.cc:2243] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.837396 32224 raft_consensus.cc:2272] T 31088ed18ff54bcea05d848f8c67e437 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.840520 32224 tablet_replica.cc:333] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.841374 32224 raft_consensus.cc:2243] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.843360 32224 raft_consensus.cc:2272] T 3b871863269841f7b0aa8baf5e640865 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.846658 32224 tablet_replica.cc:333] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
W20260709 13:15:34.841204  1566 consensus_peers.cc:597] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:34.848049 32224 raft_consensus.cc:2243] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.848899 32224 raft_consensus.cc:2272] T 90a3708334cc4bd59c73d494d2747d54 P 3bb5726887244caca146e176cc6bd070 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.848858   969 leader_election.cc:336] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:34.849725   969 leader_election.cc:304] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bb5726887244caca146e176cc6bd070; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.850591  7126 raft_consensus.cc:2749] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:34.852398 32224 tablet_replica.cc:333] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.853119 32224 raft_consensus.cc:2243] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.853650 32224 raft_consensus.cc:2272] T 6444551c84374ec98f4becf8b07446ba P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.855829 32224 tablet_replica.cc:333] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.856443 32224 raft_consensus.cc:2243] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.857558 32224 raft_consensus.cc:2272] T 9ee81d93e84c4c689b02deb555210ba4 P 3bb5726887244caca146e176cc6bd070 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.860361 32224 tablet_replica.cc:333] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.861078 32224 raft_consensus.cc:2243] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.862727 32224 raft_consensus.cc:2272] T 85bbb1b886474a538a2f1a32dd91b560 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.865661 32224 tablet_replica.cc:333] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.866497 32224 raft_consensus.cc:2243] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.867254 32224 raft_consensus.cc:2272] T ef39166ba76a41d18b3770b743ea3488 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.870287 32224 tablet_replica.cc:333] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.871101 32224 raft_consensus.cc:2243] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.871860 32224 raft_consensus.cc:2272] T 9155790402004e4f930983841dfd8c64 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.874532 32224 tablet_replica.cc:333] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.875564 32224 raft_consensus.cc:2243] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.876317 32224 raft_consensus.cc:2272] T c6e1447d2c0c4822aac226064c087c42 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.878687  1567 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:34.879515 32224 tablet_replica.cc:333] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
W20260709 13:15:34.879716  1567 consensus_peers.cc:597] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:34.880530 32224 raft_consensus.cc:2243] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.882081 32224 raft_consensus.cc:2272] T 6874f81648724ea980572af4b6fa79d2 P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.885217 32224 tablet_replica.cc:333] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.886090 32224 raft_consensus.cc:2243] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.887461 32224 raft_consensus.cc:2272] T d548ef2981894fe6972da35c997f5cff P 3bb5726887244caca146e176cc6bd070 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.890509 32224 tablet_replica.cc:333] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.891410 32224 raft_consensus.cc:2243] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.892778 32224 raft_consensus.cc:2272] T a4018887e34a40108555fd3cbaa2c975 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.895663 32224 tablet_replica.cc:333] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.896518 32224 raft_consensus.cc:2243] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.898972 32224 raft_consensus.cc:2272] T ad6ca2a4a8134a10b60d7308b9a660db P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.901902 32224 tablet_replica.cc:333] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.902565 32224 raft_consensus.cc:2243] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.903970 32224 raft_consensus.cc:2272] T 7cf198f3795b43ff951a11167bc21580 P 3bb5726887244caca146e176cc6bd070 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.906558 32224 tablet_replica.cc:333] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.907440 32224 raft_consensus.cc:2243] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.908134 32224 raft_consensus.cc:2272] T 6d9b9f004a6f4293a658cca1bb92e319 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.910271 32224 tablet_replica.cc:333] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.910820 32224 raft_consensus.cc:2243] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.911918 32224 raft_consensus.cc:2272] T bdca38d5137a4409b2e61eaf2e13fd7b P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.914074 32224 tablet_replica.cc:333] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.914096  7115 raft_consensus.cc:493] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.915022 32224 raft_consensus.cc:2243] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.914883  7115 raft_consensus.cc:515] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:34.916328 32224 raft_consensus.cc:2272] T 10b0600fc3f340eb999b12ed484fcdcb P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.919145 32224 tablet_replica.cc:333] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.919922 32224 raft_consensus.cc:2243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.921211 32224 raft_consensus.cc:2272] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 3bb5726887244caca146e176cc6bd070 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.924039  7115 leader_election.cc:290] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:34.924419 32224 tablet_replica.cc:333] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.927899  7100 raft_consensus.cc:493] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.929504  7100 raft_consensus.cc:515] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:34.929883 32224 raft_consensus.cc:2243] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.932083 32224 raft_consensus.cc:2272] T 35b46c75a1e049dd910275709bd62839 P 3bb5726887244caca146e176cc6bd070 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.932962  1566 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:34.933480  7100 leader_election.cc:290] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
I20260709 13:15:34.935560 32224 tablet_replica.cc:333] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.936391 32224 raft_consensus.cc:2243] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.937304 32224 raft_consensus.cc:2272] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.939981 32224 tablet_replica.cc:333] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.940863 32224 raft_consensus.cc:2243] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.942101 32224 raft_consensus.cc:2272] T c72e2f13b3404583a1c34e15098c42b9 P 3bb5726887244caca146e176cc6bd070 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.943575  1566 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:34.946861  1564 leader_election.cc:336] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:34.947324  1564 leader_election.cc:304] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.948138  7100 raft_consensus.cc:2749] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:34.944954 32224 tablet_replica.cc:333] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.949150 32224 raft_consensus.cc:2243] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.950383 32224 raft_consensus.cc:2272] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 3bb5726887244caca146e176cc6bd070 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:34.951486  1564 leader_election.cc:336] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:34.952047  1564 leader_election.cc:304] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:34.952778  7100 raft_consensus.cc:2749] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:34.953266 32224 tablet_replica.cc:333] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.954106 32224 raft_consensus.cc:2243] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:34.954716 32224 raft_consensus.cc:2272] T 9794c08fc6b2428b981acd6c3c238b5f P 3bb5726887244caca146e176cc6bd070 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.957206 32224 tablet_replica.cc:333] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070: stopping tablet replica
I20260709 13:15:34.957885 32224 raft_consensus.cc:2243] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:34.958928 32224 raft_consensus.cc:2272] T bedf1cefa52d44d884e14c5261c6437a P 3bb5726887244caca146e176cc6bd070 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:34.963577  7100 raft_consensus.cc:493] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:34.964587  7100 raft_consensus.cc:515] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:34.967145  7100 leader_election.cc:290] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:34.974400  1566 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:35.006929  1567 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:35.028478  1564 leader_election.cc:336] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:35.030740  1564 leader_election.cc:304] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:35.033170  7100 raft_consensus.cc:2749] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:35.043951  1567 consensus_peers.cc:597] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:35.068231  1567 consensus_peers.cc:597] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:35.078004  1566 consensus_peers.cc:597] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:35.103711  1567 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Couldn't send request to peer 3bb5726887244caca146e176cc6bd070. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:35.166580  7100 raft_consensus.cc:493] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:35.167212  7100 raft_consensus.cc:515] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.169510  7100 leader_election.cc:290] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547)
W20260709 13:15:35.181452  1566 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
W20260709 13:15:35.182212  1564 leader_election.cc:336] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:35.182585  1564 leader_election.cc:304] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 64f2cb5b0e174002811ef197f9eec829, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:35.183211  7100 raft_consensus.cc:2749] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:35.299181  1566 consensus_peers.cc:597] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:35.378123  1566 consensus_peers.cc:597] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:35.581954  1566 consensus_peers.cc:597] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:35.685123  1566 consensus_peers.cc:597] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:35.749087 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:15:35.835572  7150 raft_consensus.cc:493] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.836380  7150 raft_consensus.cc:515] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.839900  7150 leader_election.cc:290] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:35.843827  1566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111) [suppressed 635 similar messages]
I20260709 13:15:35.849079  7149 raft_consensus.cc:493] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ff541db50a49efa03669dfd777fc8e)
I20260709 13:15:35.849779  7149 raft_consensus.cc:515] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } attrs { replace: true } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.854594  7149 leader_election.cc:290] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:35.847632  1567 leader_election.cc:336] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
W20260709 13:15:35.859688  1566 leader_election.cc:336] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:35.864389  1566 leader_election.cc:304] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:35.880048  7100 raft_consensus.cc:2749] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:35.886122  1567 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:35.889882  7150 raft_consensus.cc:493] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.890811  7150 raft_consensus.cc:515] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:35.892521  1566 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:35.897898  1566 leader_election.cc:304] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:35.902258  7149 raft_consensus.cc:2749] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:35.903380  1564 leader_election.cc:336] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:35.907858  7149 raft_consensus.cc:493] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.908509  7149 raft_consensus.cc:515] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.911118  7149 leader_election.cc:290] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:35.915022  1567 leader_election.cc:336] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
W20260709 13:15:35.916306  1564 consensus_peers.cc:597] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 -> Peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Couldn't send request to peer 91ff541db50a49efa03669dfd777fc8e. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:35.922129  7100 raft_consensus.cc:493] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.922772  7100 raft_consensus.cc:515] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.925115  7100 leader_election.cc:290] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:35.900203  7150 leader_election.cc:290] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:35.931157 32224 tablet_server.cc:179] TabletServer@127.31.120.4:0 shutting down...
W20260709 13:15:35.932286  1564 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
W20260709 13:15:35.941854  1566 consensus_peers.cc:597] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:35.942343  1566 leader_election.cc:336] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:35.942667  1566 leader_election.cc:304] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:35.943464  1566 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:35.944654  7150 raft_consensus.cc:2749] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:35.946134  1567 leader_election.cc:336] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:35.946575  1567 leader_election.cc:304] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:35.947726  7150 raft_consensus.cc:2749] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:35.950661  7150 raft_consensus.cc:493] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.951251  7150 raft_consensus.cc:515] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
W20260709 13:15:35.961104  1567 leader_election.cc:336] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:35.961570  1567 leader_election.cc:304] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:35.962260  7100 raft_consensus.cc:2749] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:35.964699  1564 leader_election.cc:336] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:35.966153  7150 leader_election.cc:290] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:35.971752  1567 leader_election.cc:336] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:35.972829  1567 leader_election.cc:304] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:35.974483  7150 raft_consensus.cc:2749] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:35.992261  7150 raft_consensus.cc:493] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:35.992974  7150 raft_consensus.cc:515] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:35.996574  7150 leader_election.cc:290] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:36.002013  1564 leader_election.cc:336] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:36.003118  7150 raft_consensus.cc:493] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:36.003854  7150 raft_consensus.cc:515] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:36.009299  7150 leader_election.cc:290] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:36.010866 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:36.012123 32224 tablet_replica.cc:333] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.013218 32224 raft_consensus.cc:2243] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:36.013849  1567 leader_election.cc:336] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:36.014182 32224 raft_consensus.cc:2272] T 5b9d8e1d04724f1ba621ecf90cc49b82 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.017978 32224 tablet_replica.cc:333] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.014323  1567 leader_election.cc:304] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:36.018716  7149 raft_consensus.cc:2749] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.019879 32224 raft_consensus.cc:2243] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.021281 32224 raft_consensus.cc:2272] T 2840f4bbf21443318b9ad41dc7e3e853 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.024816 32224 tablet_replica.cc:333] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240: stopping tablet replica
W20260709 13:15:36.025393  1566 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:36.026016 32224 raft_consensus.cc:2243] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:36.026227  1567 leader_election.cc:336] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:36.027072  1567 leader_election.cc:304] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
I20260709 13:15:36.027194 32224 raft_consensus.cc:2272] T d45ee7bf01de4a3ebb2eb56dcd597d5c P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.028772  7150 raft_consensus.cc:2749] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.031077 32224 tablet_replica.cc:333] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240: stopping tablet replica
W20260709 13:15:36.026053  1566 consensus_peers.cc:597] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 -> Peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Couldn't send request to peer 64f2cb5b0e174002811ef197f9eec829. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:36.032572 32224 raft_consensus.cc:2243] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.033376 32224 raft_consensus.cc:2272] T d548ef2981894fe6972da35c997f5cff P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.038739 32224 tablet_replica.cc:333] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.039793 32224 raft_consensus.cc:2243] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.040623 32224 raft_consensus.cc:2272] T 2fb0c412ac6a42fe84d4b2ce85773e26 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.043767 32224 tablet_replica.cc:333] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.044668 32224 raft_consensus.cc:2243] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.045208 32224 raft_consensus.cc:2272] T 35b46c75a1e049dd910275709bd62839 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.048454 32224 tablet_replica.cc:333] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.049183 32224 raft_consensus.cc:2243] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.049922 32224 raft_consensus.cc:2272] T 2fe55b34a7fd4408b9a1e51ded1fd129 P 597687e464174d1ea6043161e037d240 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.050998  7150 raft_consensus.cc:493] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:36.051558  7150 raft_consensus.cc:515] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:36.053045 32224 tablet_replica.cc:333] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.053833  7150 leader_election.cc:290] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:36.053925 32224 raft_consensus.cc:2243] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.054657 32224 raft_consensus.cc:2272] T dcb3d540c3fc49bf926540f7bd349110 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.057753 32224 tablet_replica.cc:333] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.058650 32224 raft_consensus.cc:2243] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.058849  7149 raft_consensus.cc:493] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
I20260709 13:15:36.059525 32224 raft_consensus.cc:2272] T bedf1cefa52d44d884e14c5261c6437a P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.059479  7149 raft_consensus.cc:515] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:36.065313 32224 tablet_replica.cc:333] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.066222 32224 raft_consensus.cc:2243] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.067739 32224 raft_consensus.cc:2272] T c3247a08a45747539334328f4abf2acc P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:36.068605  1567 leader_election.cc:336] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:36.070775  7149 leader_election.cc:290] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
W20260709 13:15:36.075248  1564 leader_election.cc:336] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:36.075739  1564 leader_election.cc:304] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:36.076970  7149 raft_consensus.cc:2749] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.071064 32224 tablet_replica.cc:333] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.061795  7100 raft_consensus.cc:493] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3bb5726887244caca146e176cc6bd070)
W20260709 13:15:36.077674  1566 leader_election.cc:336] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:36.077778  7100 raft_consensus.cc:515] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: false } }
I20260709 13:15:36.080237  7100 leader_election.cc:290] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547), 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411)
I20260709 13:15:36.082273 32224 raft_consensus.cc:2243] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.083279 32224 raft_consensus.cc:2272] T cfd817716cdc4448b3f884eecc94351c P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:36.086364  1567 leader_election.cc:336] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:36.086547 32224 tablet_replica.cc:333] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.086932  1567 leader_election.cc:304] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 64f2cb5b0e174002811ef197f9eec829
W20260709 13:15:36.086932  1564 leader_election.cc:336] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ff541db50a49efa03669dfd777fc8e (127.31.120.2:34547): Network error: Client connection negotiation failed: client connection to 127.31.120.2:34547: connect: Connection refused (error 111)
I20260709 13:15:36.087769 32224 raft_consensus.cc:2243] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.088341  7100 raft_consensus.cc:2749] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:36.090535 32224 raft_consensus.cc:2272] T ad6ca2a4a8134a10b60d7308b9a660db P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:36.091895  1567 leader_election.cc:336] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3bb5726887244caca146e176cc6bd070 (127.31.120.3:44411): Network error: Client connection negotiation failed: client connection to 127.31.120.3:44411: connect: Connection refused (error 111)
I20260709 13:15:36.092394  1567 leader_election.cc:304] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 597687e464174d1ea6043161e037d240; no voters: 3bb5726887244caca146e176cc6bd070, 91ff541db50a49efa03669dfd777fc8e
I20260709 13:15:36.093094  7100 raft_consensus.cc:2749] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.093816 32224 tablet_replica.cc:333] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.094888 32224 raft_consensus.cc:2243] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.095854 32224 raft_consensus.cc:2272] T 10b0600fc3f340eb999b12ed484fcdcb P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.099000 32224 tablet_replica.cc:333] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.099918 32224 raft_consensus.cc:2243] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.101401 32224 raft_consensus.cc:2272] T 701b27559f8e47f9b63e5d220d3d5a58 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.104440 32224 tablet_replica.cc:333] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.105247 32224 raft_consensus.cc:2243] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.106500 32224 raft_consensus.cc:2272] T 2ec15122641447a3bd48017eb990578d P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.109577 32224 tablet_replica.cc:333] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.110571 32224 raft_consensus.cc:2243] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.111260 32224 raft_consensus.cc:2272] T c72e2f13b3404583a1c34e15098c42b9 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.114461 32224 tablet_replica.cc:333] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.115381 32224 raft_consensus.cc:2243] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.116264 32224 raft_consensus.cc:2272] T 85bbb1b886474a538a2f1a32dd91b560 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.119236 32224 tablet_replica.cc:333] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.119995 32224 raft_consensus.cc:2243] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Raft consensus shutting down.
I20260709 13:15:36.120577 32224 pending_rounds.cc:70] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Trying to abort 1 pending ops.
I20260709 13:15:36.120862 32224 pending_rounds.cc:77] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240: Aborting op as it isn't in flight: id { term: 9 index: 6 } timestamp: 7305637305330208768 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "cdfea09a06ff4100a0b80619f35f7008" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } } new_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64f2cb5b0e174002811ef197f9eec829" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43377 } attrs { replace: true } } peers { permanent_uuid: "91ff541db50a49efa03669dfd777fc8e" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 34547 } } peers { permanent_uuid: "3bb5726887244caca146e176cc6bd070" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 44411 } } peers { permanent_uuid: "597687e464174d1ea6043161e037d240" member_type: NON_VOTER last_known_addr { host: "127.31.120.4" port: 33689 } attrs { promote: true } } } }
I20260709 13:15:36.122089 32224 raft_consensus.cc:2917] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 LEARNER]: Aborting config change with OpId 9.6: Aborted: Op aborted
I20260709 13:15:36.122532 32224 raft_consensus.cc:2889] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:15:36.122846 32224 raft_consensus.cc:2272] T cdfea09a06ff4100a0b80619f35f7008 P 597687e464174d1ea6043161e037d240 [term 11 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 13:15:36.125733 32224 tablet_replica.cc:333] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.126502 32224 raft_consensus.cc:2243] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.127763 32224 raft_consensus.cc:2272] T 7de58b0d5ad94d869fac3b0491143543 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.130640 32224 tablet_replica.cc:333] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.131428 32224 raft_consensus.cc:2243] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.132354 32224 raft_consensus.cc:2272] T 7cf198f3795b43ff951a11167bc21580 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.135126 32224 tablet_replica.cc:333] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.135872 32224 raft_consensus.cc:2243] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.136858 32224 raft_consensus.cc:2272] T 9ee81d93e84c4c689b02deb555210ba4 P 597687e464174d1ea6043161e037d240 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.139789 32224 tablet_replica.cc:333] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.140558 32224 raft_consensus.cc:2243] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.141680 32224 raft_consensus.cc:2272] T 6444551c84374ec98f4becf8b07446ba P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.144456 32224 tablet_replica.cc:333] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.145202 32224 raft_consensus.cc:2243] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.146273 32224 raft_consensus.cc:2272] T 90a3708334cc4bd59c73d494d2747d54 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.148926 32224 tablet_replica.cc:333] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.149648 32224 raft_consensus.cc:2243] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.150486 32224 raft_consensus.cc:2272] T c43ab43025004edcacdb3d8a30106319 P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.153199 32224 tablet_replica.cc:333] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.153960 32224 raft_consensus.cc:2243] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.155004 32224 raft_consensus.cc:2272] T b8c347b6586f486192f48c46974a2526 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.157599 32224 tablet_replica.cc:333] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.158318 32224 raft_consensus.cc:2243] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.159348 32224 raft_consensus.cc:2272] T d3d5a097d0fe49bab1a3ec1f78f4164d P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.162040 32224 tablet_replica.cc:333] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.162802 32224 raft_consensus.cc:2243] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.163466 32224 raft_consensus.cc:2272] T 633fa6f0c6434c49aeb87667c9d97141 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.166213 32224 tablet_replica.cc:333] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.166985 32224 raft_consensus.cc:2243] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.168038 32224 raft_consensus.cc:2272] T 53d4a96b7f094cf691b31dbe6fb015bd P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.170843 32224 tablet_replica.cc:333] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.171563 32224 raft_consensus.cc:2243] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.172547 32224 raft_consensus.cc:2272] T 2b4613bf5fb0446a80ac401fe0560967 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.175266 32224 tablet_replica.cc:333] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.176028 32224 raft_consensus.cc:2243] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.177139 32224 raft_consensus.cc:2272] T 14a8781662d54452a1478c2ae0e3f911 P 597687e464174d1ea6043161e037d240 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.180012 32224 tablet_replica.cc:333] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.180753 32224 raft_consensus.cc:2243] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.181334 32224 raft_consensus.cc:2272] T 3b871863269841f7b0aa8baf5e640865 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.184042 32224 tablet_replica.cc:333] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.184772 32224 raft_consensus.cc:2243] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.185349 32224 raft_consensus.cc:2272] T 01fc410be7f74515adce264a85731b90 P 597687e464174d1ea6043161e037d240 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.188135 32224 tablet_replica.cc:333] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.188901 32224 raft_consensus.cc:2243] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.190024 32224 raft_consensus.cc:2272] T 9794c08fc6b2428b981acd6c3c238b5f P 597687e464174d1ea6043161e037d240 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.199977 32224 tablet_replica.cc:333] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.200918 32224 raft_consensus.cc:2243] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.202267 32224 raft_consensus.cc:2272] T 6cfaab49f56546a49978c6055f57640f P 597687e464174d1ea6043161e037d240 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.205037 32224 tablet_replica.cc:333] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.205740 32224 raft_consensus.cc:2243] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.206881 32224 raft_consensus.cc:2272] T 24f44707b1d3466e91403f063187a9b2 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.209549 32224 tablet_replica.cc:333] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.210270 32224 raft_consensus.cc:2243] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.211300 32224 raft_consensus.cc:2272] T 91555b4ec53349c287c5221b9030c23a P 597687e464174d1ea6043161e037d240 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.213893 32224 tablet_replica.cc:333] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.214663 32224 raft_consensus.cc:2243] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.215389 32224 raft_consensus.cc:2272] T 6533a74f2a524baab75bdfae9b5b1888 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.218238 32224 tablet_replica.cc:333] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.221510 32224 raft_consensus.cc:2243] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.222532 32224 raft_consensus.cc:2272] T 8772f5977d4b4dc6b9f7f5dcf7019b90 P 597687e464174d1ea6043161e037d240 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.225330 32224 tablet_replica.cc:333] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.226061 32224 raft_consensus.cc:2243] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.227149 32224 raft_consensus.cc:2272] T e65c03394bf04e2fb03562b27d85f5b3 P 597687e464174d1ea6043161e037d240 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.230037 32224 tablet_replica.cc:333] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.230716 32224 raft_consensus.cc:2243] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.231804 32224 raft_consensus.cc:2272] T 4997f2c7b33048a5b010a4e0c7866fa2 P 597687e464174d1ea6043161e037d240 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.234510 32224 tablet_replica.cc:333] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.235253 32224 raft_consensus.cc:2243] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.235893 32224 raft_consensus.cc:2272] T 731bda621b364aaa81d868d0cef5e739 P 597687e464174d1ea6043161e037d240 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.238751 32224 tablet_replica.cc:333] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.239466 32224 raft_consensus.cc:2243] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.240478 32224 raft_consensus.cc:2272] T c6e1447d2c0c4822aac226064c087c42 P 597687e464174d1ea6043161e037d240 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.243155 32224 tablet_replica.cc:333] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.243791 32224 raft_consensus.cc:2243] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.245003 32224 raft_consensus.cc:2272] T e0cca622722b48a7ba611a3b77376f80 P 597687e464174d1ea6043161e037d240 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.247532 32224 tablet_replica.cc:333] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.248207 32224 raft_consensus.cc:2243] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:36.249195 32224 raft_consensus.cc:2272] T 82ed8549f06d4ef3a4620f596e0139b7 P 597687e464174d1ea6043161e037d240 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.251910 32224 tablet_replica.cc:333] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240: stopping tablet replica
I20260709 13:15:36.252487 32224 raft_consensus.cc:2243] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:36.252981 32224 raft_consensus.cc:2272] T 516d927af6bf46aa947c0919f1dfc830 P 597687e464174d1ea6043161e037d240 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:36.629668 32224 tablet_server.cc:196] TabletServer@127.31.120.4:0 shutdown complete.
I20260709 13:15:36.706748 32224 master.cc:562] Master@127.31.120.62:45937 shutting down...
W20260709 13:15:36.741206   801 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:15:36.741503   801 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:36.741732   801 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:37.030290   739 catalog_manager.cc:4755] TS 64f2cb5b0e174002811ef197f9eec829 (127.31.120.1:43377): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet bedf1cefa52d44d884e14c5261c6437a: Network error: Client connection negotiation failed: client connection to 127.31.120.1:43377: connect: Connection refused (error 111)
I20260709 13:15:37.065594 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:37.066378 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:37.066766 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P ad0b4d62f81540a8878fe1d1c1552895: stopping tablet replica
I20260709 13:15:37.239712 32224 master.cc:584] Master@127.31.120.62:45937 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (199261 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:15:37.342389 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:41611
I20260709 13:15:37.343528 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:37.349661  7156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.349664  7155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.350577  7158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:37.353884 32224 server_base.cc:1061] running on GCE node
I20260709 13:15:37.356904 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:37.357139 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:37.357326 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602937357295 us; error 0 us; skew 500 ppm
I20260709 13:15:37.357842 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:37.360437 32224 webserver.cc:533] Webserver started at http://127.31.120.62:37239/ using document root <none> and password file <none>
I20260709 13:15:37.360929 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:37.361145 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:37.361430 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:37.362623 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "6aee7c93da9c4a9aacf1144b11e0129b"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.367061 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:15:37.370501  7163 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.371472 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:15:37.371843 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "6aee7c93da9c4a9aacf1144b11e0129b"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.372095 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:37.401104 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:37.402343 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:37.441550 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:41611
I20260709 13:15:37.441658  7214 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:41611 every 8 connection(s)
I20260709 13:15:37.445824  7215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:37.458561  7215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b: Bootstrap starting.
I20260709 13:15:37.464135  7215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:37.468797  7215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b: No bootstrap required, opened a new log
I20260709 13:15:37.471072  7215 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER }
I20260709 13:15:37.471544  7215 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:37.471823  7215 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aee7c93da9c4a9aacf1144b11e0129b, State: Initialized, Role: FOLLOWER
I20260709 13:15:37.472522  7215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER }
I20260709 13:15:37.472991  7215 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:37.473187  7215 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:37.473398  7215 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:37.478032  7215 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER }
I20260709 13:15:37.478572  7215 leader_election.cc:304] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aee7c93da9c4a9aacf1144b11e0129b; no voters: 
I20260709 13:15:37.479727  7215 leader_election.cc:290] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:37.480190  7218 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:37.481684  7218 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 1 LEADER]: Becoming Leader. State: Replica: 6aee7c93da9c4a9aacf1144b11e0129b, State: Running, Role: LEADER
I20260709 13:15:37.482409  7218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER }
I20260709 13:15:37.482658  7215 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:37.486891  7219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER } }
I20260709 13:15:37.486997  7220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aee7c93da9c4a9aacf1144b11e0129b. Latest consensus state: current_term: 1 leader_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aee7c93da9c4a9aacf1144b11e0129b" member_type: VOTER } }
I20260709 13:15:37.487699  7219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:37.487920  7220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:37.499722  7226 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:37.499869 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:37.504508  7226 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:37.513037  7226 catalog_manager.cc:1383] Generated new cluster ID: ffbe52e661034e3b983feb9f2e313ecf
I20260709 13:15:37.513346  7226 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:37.542693  7226 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:37.544008  7226 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:37.557199  7226 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b: Generated new TSK 0
I20260709 13:15:37.557765  7226 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:37.566171 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:37.572094  7236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.573639  7237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.576877  7239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:37.577847 32224 server_base.cc:1061] running on GCE node
I20260709 13:15:37.580101 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:37.580294 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:37.580432 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602937580421 us; error 0 us; skew 500 ppm
I20260709 13:15:37.580905 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:37.583252 32224 webserver.cc:533] Webserver started at http://127.31.120.1:36985/ using document root <none> and password file <none>
I20260709 13:15:37.583782 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:37.583993 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:37.584306 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:37.585392 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "2b2b311085b64300925884dec9a28c7d"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.589825 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:15:37.593374  7244 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.594309 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:15:37.594695 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "2b2b311085b64300925884dec9a28c7d"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.595103 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:37.620488 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:37.621693 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:37.623160 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:37.625470 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:37.625690 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.625941 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:37.626127 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.667264 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:33107
I20260709 13:15:37.667385  7306 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:33107 every 8 connection(s)
I20260709 13:15:37.672488 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:37.679525  7311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.680825  7312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.685191  7314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:37.685308 32224 server_base.cc:1061] running on GCE node
I20260709 13:15:37.687776 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:37.687978 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:37.688107 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602937688095 us; error 0 us; skew 500 ppm
I20260709 13:15:37.688560 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:37.691134 32224 webserver.cc:533] Webserver started at http://127.31.120.2:42881/ using document root <none> and password file <none>
I20260709 13:15:37.691767 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:37.691977 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:37.692263 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:37.693671 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.694242  7307 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41611
I20260709 13:15:37.694657  7307 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:37.695387  7307 heartbeater.cc:507] Master 127.31.120.62:41611 requested a full tablet report, sending...
I20260709 13:15:37.697690  7180 ts_manager.cc:194] Registered new tserver with Master: 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:37.698738 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:15:37.699533  7180 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52532
I20260709 13:15:37.702491  7319 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.703418 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:15:37.703737 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.703982 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:37.744530 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:37.745683 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:37.747052 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:37.749289 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:37.749464 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.749763 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:37.749902 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.790827 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:35887
I20260709 13:15:37.790964  7381 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:35887 every 8 connection(s)
I20260709 13:15:37.796239 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:37.804842  7385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.805006  7386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.808424  7388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:37.808845 32224 server_base.cc:1061] running on GCE node
I20260709 13:15:37.809760 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:37.809952 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:37.810181 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602937810158 us; error 0 us; skew 500 ppm
I20260709 13:15:37.810702 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:37.810827  7382 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41611
I20260709 13:15:37.811261  7382 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:37.812258  7382 heartbeater.cc:507] Master 127.31.120.62:41611 requested a full tablet report, sending...
I20260709 13:15:37.813592 32224 webserver.cc:533] Webserver started at http://127.31.120.3:36551/ using document root <none> and password file <none>
I20260709 13:15:37.814184 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:37.814383 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:37.814523  7180 ts_manager.cc:194] Registered new tserver with Master: 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:37.814666 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:37.816020 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "5270a004873440ec9a5e8106b63a6a85"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.816085  7180 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52536
I20260709 13:15:37.820639 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:15:37.824086  7393 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.824826 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:15:37.825155 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "5270a004873440ec9a5e8106b63a6a85"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.825470 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:37.838907 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:37.840205 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:37.841595 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:37.843868 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:37.844089 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.844327 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:37.844509 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.884419 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:35265
I20260709 13:15:37.884545  7455 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:35265 every 8 connection(s)
I20260709 13:15:37.889338 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:37.896404  7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.897476  7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:37.901089  7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:37.902467  7456 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41611
I20260709 13:15:37.902510 32224 server_base.cc:1061] running on GCE node
I20260709 13:15:37.903146  7456 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:37.903824 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:15:37.903980  7456 heartbeater.cc:507] Master 127.31.120.62:41611 requested a full tablet report, sending...
W20260709 13:15:37.904111 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:37.904541 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602937904523 us; error 0 us; skew 500 ppm
I20260709 13:15:37.905162 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:37.906318  7180 ts_manager.cc:194] Registered new tserver with Master: 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:37.907968  7180 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52548
I20260709 13:15:37.908032 32224 webserver.cc:533] Webserver started at http://127.31.120.4:44709/ using document root <none> and password file <none>
I20260709 13:15:37.908633 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:37.908849 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:37.909164 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:37.910261 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-3-root/instance:
uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.914486 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:15:37.917774  7467 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.918555 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:15:37.918855 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:37.919166 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:37.948745 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:37.950042 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:37.951519 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:37.953860 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:37.954104 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.954346 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:37.954528 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:37.996940 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.4:39007
I20260709 13:15:37.997058  7529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.4:39007 every 8 connection(s)
I20260709 13:15:38.011346  7530 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41611
I20260709 13:15:38.011794  7530 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:38.012465  7530 heartbeater.cc:507] Master 127.31.120.62:41611 requested a full tablet report, sending...
I20260709 13:15:38.014523  7180 ts_manager.cc:194] Registered new tserver with Master: 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:38.015223 32224 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.014788014s
I20260709 13:15:38.015960  7180 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52556
I20260709 13:15:38.041116  7180 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52568:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:15:38.291270  7345 tablet_service.cc:1511] Processing CreateTablet for tablet 961fad27981341bb99577079ea5dcc3f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:38.292495  7346 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa90396c8ee443fa7e1f1e7308e594e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:38.294341  7495 tablet_service.cc:1511] Processing CreateTablet for tablet c0c47eeb37da48cfb2366b89f2c3c628 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:38.294945  7492 tablet_service.cc:1511] Processing CreateTablet for tablet 2118345d07be4c3195f476caf906740a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:38.291265  7347 tablet_service.cc:1511] Processing CreateTablet for tablet c0c47eeb37da48cfb2366b89f2c3c628 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:38.295783  7495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c47eeb37da48cfb2366b89f2c3c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.298274  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 961fad27981341bb99577079ea5dcc3f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:38.299083  7493 tablet_service.cc:1511] Processing CreateTablet for tablet 55b3b562b9fd453ab8b640d44359747a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:38.299749  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961fad27981341bb99577079ea5dcc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.300521  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b3b562b9fd453ab8b640d44359747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.303785  7270 tablet_service.cc:1511] Processing CreateTablet for tablet 55b3b562b9fd453ab8b640d44359747a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:38.305253  7270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b3b562b9fd453ab8b640d44359747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.306665  7344 tablet_service.cc:1511] Processing CreateTablet for tablet c92a5d8ff52e4a238b88b5dd6ed2317d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:38.308777  7269 tablet_service.cc:1511] Processing CreateTablet for tablet c92a5d8ff52e4a238b88b5dd6ed2317d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:38.309028  7272 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa90396c8ee443fa7e1f1e7308e594e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:38.310133  7269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92a5d8ff52e4a238b88b5dd6ed2317d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.312659  7343 tablet_service.cc:1511] Processing CreateTablet for tablet 2118345d07be4c3195f476caf906740a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:38.314514  7268 tablet_service.cc:1511] Processing CreateTablet for tablet 2118345d07be4c3195f476caf906740a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:38.315115  7342 tablet_service.cc:1511] Processing CreateTablet for tablet ad2b493283374c639936c29e00cdd96c (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:38.315573  7421 tablet_service.cc:1511] Processing CreateTablet for tablet c0c47eeb37da48cfb2366b89f2c3c628 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:38.317008  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c47eeb37da48cfb2366b89f2c3c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.316802  7267 tablet_service.cc:1511] Processing CreateTablet for tablet 1abce17e47154429b9f10b822f48ab3e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:38.318228  7267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abce17e47154429b9f10b822f48ab3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.320132  7266 tablet_service.cc:1511] Processing CreateTablet for tablet de79d0d3ecc64a7a96c61be7604e6753 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:38.320817  7420 tablet_service.cc:1511] Processing CreateTablet for tablet 2aa90396c8ee443fa7e1f1e7308e594e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:38.322225  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa90396c8ee443fa7e1f1e7308e594e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.324527  7419 tablet_service.cc:1511] Processing CreateTablet for tablet 55b3b562b9fd453ab8b640d44359747a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:38.325850  7419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b3b562b9fd453ab8b640d44359747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.304812  7494 tablet_service.cc:1511] Processing CreateTablet for tablet 961fad27981341bb99577079ea5dcc3f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:38.328666  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961fad27981341bb99577079ea5dcc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.328763  7418 tablet_service.cc:1511] Processing CreateTablet for tablet c92a5d8ff52e4a238b88b5dd6ed2317d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:38.330173  7418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92a5d8ff52e4a238b88b5dd6ed2317d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.332293  7266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79d0d3ecc64a7a96c61be7604e6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.332799  7417 tablet_service.cc:1511] Processing CreateTablet for tablet 1abce17e47154429b9f10b822f48ab3e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:38.333439  7272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa90396c8ee443fa7e1f1e7308e594e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.334136  7417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abce17e47154429b9f10b822f48ab3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.336112  7268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2118345d07be4c3195f476caf906740a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.336831  7416 tablet_service.cc:1511] Processing CreateTablet for tablet ad2b493283374c639936c29e00cdd96c (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:38.338191  7416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2b493283374c639936c29e00cdd96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.341838  7491 tablet_service.cc:1511] Processing CreateTablet for tablet 1abce17e47154429b9f10b822f48ab3e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:38.342829  7487 tablet_service.cc:1511] Processing CreateTablet for tablet 99f8dd60683a4a94b2eab776a941327a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:38.345103  7490 tablet_service.cc:1511] Processing CreateTablet for tablet ad2b493283374c639936c29e00cdd96c (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:38.346463  7415 tablet_service.cc:1511] Processing CreateTablet for tablet de79d0d3ecc64a7a96c61be7604e6753 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:38.348006  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 97a88676349042cda17ab04278fa8b6f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:38.350376  7413 tablet_service.cc:1511] Processing CreateTablet for tablet 99f8dd60683a4a94b2eab776a941327a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:38.351083  7488 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad3d31ca56b45dda55d0616cc89204b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:38.353080  7412 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd6aaff170d478f8d6920b5849c6978 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:38.348747  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad3d31ca56b45dda55d0616cc89204b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:38.353996  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 141286c286284003b795291ad8766165 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:38.355165  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79d0d3ecc64a7a96c61be7604e6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.356092  7412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd6aaff170d478f8d6920b5849c6978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.356705  7413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99f8dd60683a4a94b2eab776a941327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.358767  7341 tablet_service.cc:1511] Processing CreateTablet for tablet de79d0d3ecc64a7a96c61be7604e6753 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:38.370018  7338 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd6aaff170d478f8d6920b5849c6978 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:38.360004  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad3d31ca56b45dda55d0616cc89204b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.362056  7340 tablet_service.cc:1511] Processing CreateTablet for tablet 97a88676349042cda17ab04278fa8b6f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:38.363879  7339 tablet_service.cc:1511] Processing CreateTablet for tablet 99f8dd60683a4a94b2eab776a941327a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:38.292924  7345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961fad27981341bb99577079ea5dcc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.381279  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c47eeb37da48cfb2366b89f2c3c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.381983  7342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2b493283374c639936c29e00cdd96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.382634  7339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99f8dd60683a4a94b2eab776a941327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.383335  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92a5d8ff52e4a238b88b5dd6ed2317d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.384717  7340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a88676349042cda17ab04278fa8b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.387991  7343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2118345d07be4c3195f476caf906740a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.399955  7550 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:38.393525  7346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aa90396c8ee443fa7e1f1e7308e594e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.416179  7341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79d0d3ecc64a7a96c61be7604e6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.397460  7491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abce17e47154429b9f10b822f48ab3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.440889  7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd6aaff170d478f8d6920b5849c6978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.445389  7413 tablet_service.cc:1511] Processing CreateTablet for tablet 271a5a5acaf443f6ae8e2122f6f16166 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:38.446934  7413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271a5a5acaf443f6ae8e2122f6f16166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.448833  7265 tablet_service.cc:1511] Processing CreateTablet for tablet 97a88676349042cda17ab04278fa8b6f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:38.450290  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a88676349042cda17ab04278fa8b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.453339  7264 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad3d31ca56b45dda55d0616cc89204b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:38.454841  7264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad3d31ca56b45dda55d0616cc89204b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.456930  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd6aaff170d478f8d6920b5849c6978 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:38.458345  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd6aaff170d478f8d6920b5849c6978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.462181  7492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2118345d07be4c3195f476caf906740a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.462617  7551 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:38.468020  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141286c286284003b795291ad8766165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.473202  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2b493283374c639936c29e00cdd96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.474190  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a88676349042cda17ab04278fa8b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.484653  7488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad3d31ca56b45dda55d0616cc89204b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.487417  7487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99f8dd60683a4a94b2eab776a941327a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.489319  7416 tablet_service.cc:1511] Processing CreateTablet for tablet 270e9deedc904538b2b42381a2aa3889 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:38.490371  7550 tablet_bootstrap.cc:654] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:38.491118  7416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270e9deedc904538b2b42381a2aa3889. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:38.494549  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:38.499953  7495 tablet_service.cc:1511] Processing CreateTablet for tablet 343f79626b484d4aa20e247aac64c837 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:38.502020  7495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343f79626b484d4aa20e247aac64c837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.508796  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 343f79626b484d4aa20e247aac64c837 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:38.536273  7551 tablet_bootstrap.cc:654] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:38.577561  7550 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:38.578135  7550 ts_tablet_manager.cc:1403] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.179s	user 0.020s	sys 0.000s
I20260709 13:15:38.581696  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343f79626b484d4aa20e247aac64c837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.590183  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 343f79626b484d4aa20e247aac64c837 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:38.592598  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343f79626b484d4aa20e247aac64c837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.608806  7269 tablet_service.cc:1511] Processing CreateTablet for tablet 141286c286284003b795291ad8766165 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:38.611344  7269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141286c286284003b795291ad8766165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.596961  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 271a5a5acaf443f6ae8e2122f6f16166 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:38.599216  7550 raft_consensus.cc:359] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:38.619808  7550 raft_consensus.cc:385] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:38.620160  7550 raft_consensus.cc:740] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:38.621009  7550 consensus_queue.cc:260] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:38.621275  7413 tablet_service.cc:1511] Processing CreateTablet for tablet be0584005e4e4271a39fc114468c717b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:38.601123  7345 tablet_service.cc:1511] Processing CreateTablet for tablet 141286c286284003b795291ad8766165 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:38.579082  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 56bce1894cd442d5bc2580cb8d02012d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:38.603363  7552 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:38.606611  7338 tablet_service.cc:1511] Processing CreateTablet for tablet 271a5a5acaf443f6ae8e2122f6f16166 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:38.609251  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 56bce1894cd442d5bc2580cb8d02012d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:38.607434  7417 tablet_service.cc:1511] Processing CreateTablet for tablet 0079a4cd3d134d079f559b4073824860 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:38.602486  7270 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1a55fbf26d4f84bafaab065d4feece (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:38.625836  7265 tablet_service.cc:1511] Processing CreateTablet for tablet be0584005e4e4271a39fc114468c717b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:38.634752  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 56bce1894cd442d5bc2580cb8d02012d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:38.635799  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bce1894cd442d5bc2580cb8d02012d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.642031  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0584005e4e4271a39fc114468c717b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.646994  7267 tablet_service.cc:1511] Processing CreateTablet for tablet 270e9deedc904538b2b42381a2aa3889 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:38.647859  7413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0584005e4e4271a39fc114468c717b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.648358  7267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270e9deedc904538b2b42381a2aa3889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.651579  7417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0079a4cd3d134d079f559b4073824860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.653138  7554 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:38.662416  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271a5a5acaf443f6ae8e2122f6f16166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.668000  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bce1894cd442d5bc2580cb8d02012d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.674612  7344 tablet_service.cc:1511] Processing CreateTablet for tablet 0079a4cd3d134d079f559b4073824860 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:38.624269  7492 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1a55fbf26d4f84bafaab065d4feece (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:38.675957  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bce1894cd442d5bc2580cb8d02012d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.676414  7492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1a55fbf26d4f84bafaab065d4feece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.679828  7345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141286c286284003b795291ad8766165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.680410  7552 tablet_bootstrap.cc:654] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:38.635226  7488 tablet_service.cc:1511] Processing CreateTablet for tablet 0079a4cd3d134d079f559b4073824860 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:38.689842  7551 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:38.693522  7551 ts_tablet_manager.cc:1403] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.231s	user 0.009s	sys 0.004s
I20260709 13:15:38.696691  7550 ts_tablet_manager.cc:1434] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.118s	user 0.020s	sys 0.037s
I20260709 13:15:38.691702  7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271a5a5acaf443f6ae8e2122f6f16166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.700675  7493 tablet_service.cc:1511] Processing CreateTablet for tablet 598d8f846ffc44fcaca77f1973eb6b7b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:38.701574  7342 tablet_service.cc:1511] Processing CreateTablet for tablet 270e9deedc904538b2b42381a2aa3889 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:38.702103  7307 heartbeater.cc:499] Master 127.31.120.62:41611 was elected leader, sending a full tablet report...
I20260709 13:15:38.702880  7342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270e9deedc904538b2b42381a2aa3889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.704602  7268 tablet_service.cc:1511] Processing CreateTablet for tablet 598d8f846ffc44fcaca77f1973eb6b7b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:38.705255  7346 tablet_service.cc:1511] Processing CreateTablet for tablet be0584005e4e4271a39fc114468c717b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:38.705997  7268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 598d8f846ffc44fcaca77f1973eb6b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.706593  7346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0584005e4e4271a39fc114468c717b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.702164  7488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0079a4cd3d134d079f559b4073824860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.703545  7415 tablet_service.cc:1511] Processing CreateTablet for tablet 942a30e040ee4dcca90e52f6dc78a64d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:38.714337  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942a30e040ee4dcca90e52f6dc78a64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.715827  7551 raft_consensus.cc:359] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:38.716720  7551 raft_consensus.cc:385] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:38.717063  7551 raft_consensus.cc:740] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:38.717895  7340 tablet_service.cc:1511] Processing CreateTablet for tablet 598d8f846ffc44fcaca77f1973eb6b7b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:38.723990  7551 consensus_queue.cc:260] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:38.740522  7530 heartbeater.cc:499] Master 127.31.120.62:41611 was elected leader, sending a full tablet report...
I20260709 13:15:38.740764  7270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1a55fbf26d4f84bafaab065d4feece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.695112  7339 tablet_service.cc:1511] Processing CreateTablet for tablet 5a1a55fbf26d4f84bafaab065d4feece (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260709 13:15:38.752019  7531 tablet.cc:2406] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:38.744140  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0079a4cd3d134d079f559b4073824860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.763125  7272 tablet_service.cc:1511] Processing CreateTablet for tablet 942a30e040ee4dcca90e52f6dc78a64d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:38.765030  7272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942a30e040ee4dcca90e52f6dc78a64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.772316  7554 tablet_bootstrap.cc:654] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:38.772953  7490 tablet_service.cc:1511] Processing CreateTablet for tablet 942a30e040ee4dcca90e52f6dc78a64d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:38.777722  7340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 598d8f846ffc44fcaca77f1973eb6b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.802531  7339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a1a55fbf26d4f84bafaab065d4feece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.804819  7556 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.811184  7550 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:38.820351  7382 heartbeater.cc:499] Master 127.31.120.62:41611 was elected leader, sending a full tablet report...
I20260709 13:15:38.822731  7416 tablet_service.cc:1511] Processing CreateTablet for tablet e1e566e4da5c4b478489086154cbbd13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:38.824585  7487 tablet_service.cc:1511] Processing CreateTablet for tablet e1e566e4da5c4b478489086154cbbd13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:38.824892  7343 tablet_service.cc:1511] Processing CreateTablet for tablet f0ae5b6a026546828ed9671f9ca41aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:38.826021  7416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e566e4da5c4b478489086154cbbd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.826318  7343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ae5b6a026546828ed9671f9ca41aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.826409  7341 tablet_service.cc:1511] Processing CreateTablet for tablet e1e566e4da5c4b478489086154cbbd13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:38.827945  7341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e566e4da5c4b478489086154cbbd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.814749  7556 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:38.833114  7417 tablet_service.cc:1511] Processing CreateTablet for tablet f0ae5b6a026546828ed9671f9ca41aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:38.835366  7494 tablet_service.cc:1511] Processing CreateTablet for tablet 76381a75f921496eb6274bd7df95f601 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:38.838335  7417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ae5b6a026546828ed9671f9ca41aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.839526  7556 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:38.856529  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 598d8f846ffc44fcaca77f1973eb6b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.860026  7347 tablet_service.cc:1511] Processing CreateTablet for tablet 76381a75f921496eb6274bd7df95f601 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:38.861403  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76381a75f921496eb6274bd7df95f601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.863945  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76381a75f921496eb6274bd7df95f601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.866019  7487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e566e4da5c4b478489086154cbbd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.867942  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942a30e040ee4dcca90e52f6dc78a64d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.869787  7267 tablet_service.cc:1511] Processing CreateTablet for tablet f0ae5b6a026546828ed9671f9ca41aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:38.875470  7267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ae5b6a026546828ed9671f9ca41aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.883087  7554 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:38.883708  7554 ts_tablet_manager.cc:1403] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.231s	user 0.010s	sys 0.013s
I20260709 13:15:38.886739  7554 raft_consensus.cc:359] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:38.887462  7554 raft_consensus.cc:385] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:38.887790  7554 raft_consensus.cc:740] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:38.888590  7554 consensus_queue.cc:260] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:38.874251  7551 ts_tablet_manager.cc:1434] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.180s	user 0.040s	sys 0.016s
I20260709 13:15:38.904336  7269 tablet_service.cc:1511] Processing CreateTablet for tablet 04afbc5d731346f9a3c99d9d89f097f0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:38.907245  7269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04afbc5d731346f9a3c99d9d89f097f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.916476  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 76381a75f921496eb6274bd7df95f601 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:38.917878  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76381a75f921496eb6274bd7df95f601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.924269  7456 heartbeater.cc:499] Master 127.31.120.62:41611 was elected leader, sending a full tablet report...
I20260709 13:15:38.919675  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 04afbc5d731346f9a3c99d9d89f097f0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:38.925559  7552 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:38.927395  7552 ts_tablet_manager.cc:1403] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.328s	user 0.021s	sys 0.026s
I20260709 13:15:38.929790  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd43c082a974fbe87828b19abd5bc02 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:38.925747  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04afbc5d731346f9a3c99d9d89f097f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.936051  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd43c082a974fbe87828b19abd5bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.959968  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:38.964181  7340 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd43c082a974fbe87828b19abd5bc02 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:15:38.964799  7471 leader_election.cc:343] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:38.965613  7340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd43c082a974fbe87828b19abd5bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.974090  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:38.975947  7468 leader_election.cc:343] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:38.976431  7468 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:38.977232  7556 raft_consensus.cc:2749] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:38.986119  7551 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:38.962404  7552 raft_consensus.cc:359] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:38.988310  7552 raft_consensus.cc:385] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.001109  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd43c082a974fbe87828b19abd5bc02 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:39.003073  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd43c082a974fbe87828b19abd5bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.008744  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 46e82c6ed54a49e39710ec850e7ac826 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:39.010259  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e82c6ed54a49e39710ec850e7ac826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:38.988631  7552 raft_consensus.cc:740] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.027756  7565 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.028415  7565 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.030016  7338 tablet_service.cc:1511] Processing CreateTablet for tablet 46e82c6ed54a49e39710ec850e7ac826 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:39.031208  7565 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:39.031404  7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e82c6ed54a49e39710ec850e7ac826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.052091  7558 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.052678  7558 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.055279  7558 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:39.031672  7552 consensus_queue.cc:260] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.076439  7554 ts_tablet_manager.cc:1434] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.192s	user 0.031s	sys 0.034s
I20260709 13:15:39.095567  7550 tablet_bootstrap.cc:654] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.121901  7551 tablet_bootstrap.cc:654] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.081842  7487 tablet_service.cc:1511] Processing CreateTablet for tablet 04afbc5d731346f9a3c99d9d89f097f0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:39.137444  7341 tablet_service.cc:1511] Processing CreateTablet for tablet 623d8610e992488e9302c20596f4a302 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:39.136840  7490 tablet_service.cc:1511] Processing CreateTablet for tablet 623d8610e992488e9302c20596f4a302 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:39.149976  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 623d8610e992488e9302c20596f4a302 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:39.151326  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623d8610e992488e9302c20596f4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.155272  7552 ts_tablet_manager.cc:1434] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.228s	user 0.009s	sys 0.006s
I20260709 13:15:39.156385  7552 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:39.161594  7554 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:39.164186  7487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04afbc5d731346f9a3c99d9d89f097f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.164842  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623d8610e992488e9302c20596f4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.169238  7341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623d8610e992488e9302c20596f4a302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.171150  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:39.172469  7346 tablet_service.cc:1511] Processing CreateTablet for tablet 90ec340d335041b0bc58e0bcdf0b8970 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
W20260709 13:15:39.172940  7320 leader_election.cc:343] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.174558  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:39.178589  7397 leader_election.cc:343] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.174193  7346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ec340d335041b0bc58e0bcdf0b8970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.181035  7269 tablet_service.cc:1511] Processing CreateTablet for tablet b6e534925d6a411990fea3bbc14a4935 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:39.182684  7269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e534925d6a411990fea3bbc14a4935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.183885  7489 tablet_service.cc:1511] Processing CreateTablet for tablet b6e534925d6a411990fea3bbc14a4935 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:39.186880  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:39.188519  7320 leader_election.cc:343] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.189035  7320 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:39.190024  7564 raft_consensus.cc:2749] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.190804  7552 tablet_bootstrap.cc:654] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.195226  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e534925d6a411990fea3bbc14a4935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.195751  7550 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:39.196264  7550 ts_tablet_manager.cc:1403] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.391s	user 0.010s	sys 0.007s
I20260709 13:15:39.197422  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 46e82c6ed54a49e39710ec850e7ac826 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:39.197845  7413 tablet_service.cc:1511] Processing CreateTablet for tablet b6e534925d6a411990fea3bbc14a4935 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:39.198704  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e82c6ed54a49e39710ec850e7ac826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.199333  7413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e534925d6a411990fea3bbc14a4935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.192219  7554 tablet_bootstrap.cc:654] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.199319  7550 raft_consensus.cc:359] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.206419  7550 raft_consensus.cc:385] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.206741  7550 raft_consensus.cc:740] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.212431  7550 consensus_queue.cc:260] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.217149  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 90ec340d335041b0bc58e0bcdf0b8970 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:39.218513  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ec340d335041b0bc58e0bcdf0b8970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.218609  7550 ts_tablet_manager.cc:1434] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:15:39.222437  7494 tablet_service.cc:1511] Processing CreateTablet for tablet 90ec340d335041b0bc58e0bcdf0b8970 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:39.224277  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90ec340d335041b0bc58e0bcdf0b8970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.232445  7415 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:39.233947  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.235618  7550 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:39.237635  7551 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:39.238204  7551 ts_tablet_manager.cc:1403] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.252s	user 0.012s	sys 0.002s
I20260709 13:15:39.241257  7270 tablet_service.cc:1511] Processing CreateTablet for tablet 239bdb3893f6422a9456da1eec09ac9e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:39.242713  7270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239bdb3893f6422a9456da1eec09ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.246014  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:39.247227  7420 tablet_service.cc:1511] Processing CreateTablet for tablet 178ebd1ec9704b868e8dec3ccdfad30d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:39.247529  7490 tablet_service.cc:1511] Processing CreateTablet for tablet 239bdb3893f6422a9456da1eec09ac9e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:39.248728  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178ebd1ec9704b868e8dec3ccdfad30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.248473  7338 tablet_service.cc:1511] Processing CreateTablet for tablet 239bdb3893f6422a9456da1eec09ac9e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:39.249079  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239bdb3893f6422a9456da1eec09ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:39.255061  7394 leader_election.cc:343] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.258276  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 178ebd1ec9704b868e8dec3ccdfad30d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:39.260304  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178ebd1ec9704b868e8dec3ccdfad30d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.264097  7394 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.263646  7571 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.267431  7558 raft_consensus.cc:2749] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.268005  7571 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.270298  7550 tablet_bootstrap.cc:654] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.270895  7571 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.289041  7416 tablet_service.cc:1511] Processing CreateTablet for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:39.290488  7416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.291793  7339 tablet_service.cc:1511] Processing CreateTablet for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:39.291703  7493 tablet_service.cc:1511] Processing CreateTablet for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:39.297536  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.298527  7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239bdb3893f6422a9456da1eec09ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.241240  7551 raft_consensus.cc:359] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.297629  7340 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:39.307905  7339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57da519e6ecb4abbaf3b5361a4e0b9ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.296325  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:39.312913  7551 raft_consensus.cc:385] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.324096  7340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.326270  7552 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:39.326864  7552 ts_tablet_manager.cc:1403] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.171s	user 0.019s	sys 0.010s
I20260709 13:15:39.329948  7552 raft_consensus.cc:359] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.330668  7552 raft_consensus.cc:385] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.330976  7552 raft_consensus.cc:740] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.331913  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8365f37a3b4cdaa6fdfd3fb9f9e35f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.334400  7341 tablet_service.cc:1511] Processing CreateTablet for tablet ad9e725766854f9f8f7ab7741e09179b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:39.317104  7551 raft_consensus.cc:740] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.335934  7341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad9e725766854f9f8f7ab7741e09179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.337572  7554 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:39.338128  7554 ts_tablet_manager.cc:1403] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.193s	user 0.014s	sys 0.035s
I20260709 13:15:39.339417  7551 consensus_queue.cc:260] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.349028  7551 ts_tablet_manager.cc:1434] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.111s	user 0.005s	sys 0.004s
I20260709 13:15:39.350128  7551 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:39.352114  7552 consensus_queue.cc:260] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.353879  7556 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.354955  7556 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.356130  7552 ts_tablet_manager.cc:1434] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:15:39.357129  7552 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:39.359951  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.360916  7556 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.362396  7269 tablet_service.cc:1511] Processing CreateTablet for tablet 178ebd1ec9704b868e8dec3ccdfad30d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:39.362743  7263 tablet_service.cc:1511] Processing CreateTablet for tablet ad9e725766854f9f8f7ab7741e09179b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:39.364916  7269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178ebd1ec9704b868e8dec3ccdfad30d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:39.365185  7471 leader_election.cc:343] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.365535  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad9e725766854f9f8f7ab7741e09179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.366113  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.378021  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
W20260709 13:15:39.379578  7468 leader_election.cc:343] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.380048  7468 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.383832  7556 raft_consensus.cc:2749] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.385816  7413 tablet_service.cc:1511] Processing CreateTablet for tablet ad9e725766854f9f8f7ab7741e09179b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:39.387298  7413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad9e725766854f9f8f7ab7741e09179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.389055  7493 tablet_service.cc:1511] Processing CreateTablet for tablet aeedebd8085f46878c1a27393ea26f13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:39.390412  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeedebd8085f46878c1a27393ea26f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.390829  7270 tablet_service.cc:1511] Processing CreateTablet for tablet aeedebd8085f46878c1a27393ea26f13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:39.392187  7270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeedebd8085f46878c1a27393ea26f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.343035  7554 raft_consensus.cc:359] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.394793  7554 raft_consensus.cc:385] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.395139  7554 raft_consensus.cc:740] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.399255  7554 consensus_queue.cc:260] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.400341  7357 raft_consensus.cc:2468] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:39.402313  7551 tablet_bootstrap.cc:654] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.403940  7248 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:39.404824  7571 raft_consensus.cc:2804] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:39.405187  7571 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.405517  7571 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.406845  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:39.409566  7245 leader_election.cc:343] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.410589  7552 tablet_bootstrap.cc:654] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.410424  7346 tablet_service.cc:1511] Processing CreateTablet for tablet aeedebd8085f46878c1a27393ea26f13 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:39.411804  7346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeedebd8085f46878c1a27393ea26f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.412997  7344 tablet_service.cc:1511] Processing CreateTablet for tablet d4eac7e41e3a46a2aed7ecb29109d99d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:39.414259  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eac7e41e3a46a2aed7ecb29109d99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.417932  7342 tablet_service.cc:1511] Processing CreateTablet for tablet 9d76f633b67148ef905cfa3b1d5c590e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:39.418241  7420 tablet_service.cc:1511] Processing CreateTablet for tablet d4eac7e41e3a46a2aed7ecb29109d99d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:39.419142  7550 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:39.419324  7342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d76f633b67148ef905cfa3b1d5c590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.419706  7550 ts_tablet_manager.cc:1403] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.200s	user 0.011s	sys 0.006s
I20260709 13:15:39.421602  7272 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ac1115f84af29bb612331df632a6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:39.421945  7268 tablet_service.cc:1511] Processing CreateTablet for tablet 9d76f633b67148ef905cfa3b1d5c590e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:39.422930  7272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ac1115f84af29bb612331df632a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.419544  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eac7e41e3a46a2aed7ecb29109d99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.425431  7347 tablet_service.cc:1511] Processing CreateTablet for tablet 27a84c07d08e47da9a71cd1c513c6105 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:39.426751  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a84c07d08e47da9a71cd1c513c6105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.426838  7268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d76f633b67148ef905cfa3b1d5c590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.436101  7550 raft_consensus.cc:359] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.438513  7550 raft_consensus.cc:385] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.438737  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 9d76f633b67148ef905cfa3b1d5c590e (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:39.439010  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3fe3d05909404e9c7e64e0f20ba873 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:39.440230  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d76f633b67148ef905cfa3b1d5c590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.438802  7550 raft_consensus.cc:740] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.441627  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3fe3d05909404e9c7e64e0f20ba873 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:39.442194  7490 tablet_service.cc:1511] Processing CreateTablet for tablet 27a84c07d08e47da9a71cd1c513c6105 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:39.443050  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3fe3d05909404e9c7e64e0f20ba873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.443845  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a84c07d08e47da9a71cd1c513c6105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.444707  7265 tablet_service.cc:1511] Processing CreateTablet for tablet 27a84c07d08e47da9a71cd1c513c6105 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:39.442194  7486 tablet_service.cc:1511] Processing CreateTablet for tablet d4eac7e41e3a46a2aed7ecb29109d99d (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:39.446120  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a84c07d08e47da9a71cd1c513c6105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.447431  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eac7e41e3a46a2aed7ecb29109d99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.451922  7550 consensus_queue.cc:260] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.454272  7550 ts_tablet_manager.cc:1434] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.002s
I20260709 13:15:39.459971  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3fe3d05909404e9c7e64e0f20ba873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.468498  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3fe3d05909404e9c7e64e0f20ba873 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:39.469940  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3fe3d05909404e9c7e64e0f20ba873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.471185  7551 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:39.482373  7551 ts_tablet_manager.cc:1403] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.000s
I20260709 13:15:39.478011  7552 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:39.486518  7552 ts_tablet_manager.cc:1403] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.002s
I20260709 13:15:39.489492  7552 raft_consensus.cc:359] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.490226  7552 raft_consensus.cc:385] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.490533  7552 raft_consensus.cc:740] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.474938  7347 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec7093b8e1740cba58723ac7119486b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:39.491304  7551 raft_consensus.cc:359] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.497131  7551 raft_consensus.cc:385] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.492031  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec7093b8e1740cba58723ac7119486b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.497561  7551 raft_consensus.cc:740] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.498610  7551 consensus_queue.cc:260] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.501847  7571 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.505105  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:39.505800  7357 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.507128  7551 ts_tablet_manager.cc:1434] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
W20260709 13:15:39.497242  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:39.484023  7554 ts_tablet_manager.cc:1434] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.146s	user 0.011s	sys 0.000s
I20260709 13:15:39.469723  7415 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ac1115f84af29bb612331df632a6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:39.517647  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ac1115f84af29bb612331df632a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.518991  7551 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:39.500847  7550 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:39.523733  7494 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8ac1115f84af29bb612331df632a6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:39.525053  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8ac1115f84af29bb612331df632a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.527769  7554 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:39.529426  7551 tablet_bootstrap.cc:654] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.519701  7571 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.493503  7552 consensus_queue.cc:260] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.534672  7552 ts_tablet_manager.cc:1434] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 13:15:39.545195  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
W20260709 13:15:39.547013  7245 leader_election.cc:343] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.556279  7552 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:39.531984  7571 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.561046  7580 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.519281  7357 raft_consensus.cc:2468] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:39.561596  7580 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.562811  7550 tablet_bootstrap.cc:654] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.566102  7554 tablet_bootstrap.cc:654] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.566640  7248 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:39.558794  7571 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.582203  7420 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec7093b8e1740cba58723ac7119486b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:39.583361  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec7093b8e1740cba58723ac7119486b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:39.616518  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec7093b8e1740cba58723ac7119486b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.618505  7554 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:39.619186  7554 ts_tablet_manager.cc:1403] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.003s
I20260709 13:15:39.620589  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec7093b8e1740cba58723ac7119486b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.588851  7585 raft_consensus.cc:2804] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:39.592377  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:39.623917  7551 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:39.592549  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.597131  7265 tablet_service.cc:1511] Processing CreateTablet for tablet ce24f22b1cbe4955a186eb1a28aa32bf (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
W20260709 13:15:39.624176  7245 leader_election.cc:343] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.624506  7551 ts_tablet_manager.cc:1403] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.107s	user 0.058s	sys 0.031s
I20260709 13:15:39.625087  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce24f22b1cbe4955a186eb1a28aa32bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.626045  7554 raft_consensus.cc:359] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.626791  7554 raft_consensus.cc:385] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.627161  7554 raft_consensus.cc:740] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.599352  7344 tablet_service.cc:1511] Processing CreateTablet for tablet ce24f22b1cbe4955a186eb1a28aa32bf (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:39.627529  7551 raft_consensus.cc:359] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.628301  7551 raft_consensus.cc:385] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.628408  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce24f22b1cbe4955a186eb1a28aa32bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.628072  7554 consensus_queue.cc:260] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.628625  7551 raft_consensus.cc:740] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.630329  7554 ts_tablet_manager.cc:1434] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
I20260709 13:15:39.600392  7580 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:39.631138  7551 consensus_queue.cc:260] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.602506  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:39.604864  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.606568  7571 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:39.610570  7550 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:39.612977  7552 tablet_bootstrap.cc:654] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.633517  7356 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:39.633895  7550 ts_tablet_manager.cc:1403] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.012s
W20260709 13:15:39.635349  7245 leader_election.cc:343] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.636639  7248 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:39.637075  7550 raft_consensus.cc:359] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.637885  7550 raft_consensus.cc:385] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.638175  7550 raft_consensus.cc:740] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.638260  7571 raft_consensus.cc:2804] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:39.638806  7571 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.639207  7571 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.639081  7550 consensus_queue.cc:260] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.641464  7550 ts_tablet_manager.cc:1434] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:15:39.642536  7550 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:39.648778  7550 tablet_bootstrap.cc:654] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.653273  7490 tablet_service.cc:1511] Processing CreateTablet for tablet fc3465e298174cd9b6cdf4a7ea730ebe (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:39.654651  7490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3465e298174cd9b6cdf4a7ea730ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.656574  7551 ts_tablet_manager.cc:1434] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:15:39.657703  7551 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:39.658330  7357 raft_consensus.cc:2468] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:39.659638  7248 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:39.660460  7580 raft_consensus.cc:2804] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:39.660825  7580 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.661139  7580 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.657387  7571 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.672030  7347 tablet_service.cc:1511] Processing CreateTablet for tablet fc3465e298174cd9b6cdf4a7ea730ebe (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:39.677335  7551 tablet_bootstrap.cc:654] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.673527  7585 raft_consensus.cc:697] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:39.682156  7415 tablet_service.cc:1511] Processing CreateTablet for tablet ce24f22b1cbe4955a186eb1a28aa32bf (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:39.685154  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3465e298174cd9b6cdf4a7ea730ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.686007  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 2eb1ec31ba3340fb815cc11f5c116aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:39.690330  7552 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:39.684178  7554 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:39.696655  7265 tablet_service.cc:1511] Processing CreateTablet for tablet 2eb1ec31ba3340fb815cc11f5c116aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:39.698294  7585 consensus_queue.cc:237] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.701930  7558 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.702447  7558 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.702872  7552 ts_tablet_manager.cc:1403] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.008s
I20260709 13:15:39.700374  7271 tablet_service.cc:1511] Processing CreateTablet for tablet fc3465e298174cd9b6cdf4a7ea730ebe (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:39.704849  7558 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:39.706260  7552 raft_consensus.cc:359] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.693019  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce24f22b1cbe4955a186eb1a28aa32bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.693456  7580 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.717555  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:39.717959  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:39.718609  7357 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.710076  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:39.718899  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.700297  7265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eb1ec31ba3340fb815cc11f5c116aa0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:39.720321  7394 leader_election.cc:343] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.721312  7552 raft_consensus.cc:385] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.722447  7556 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.727922  7556 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.728181  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:39.728181  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:15:39.730603  7245 leader_election.cc:343] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.731125  7356 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.731980  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.732613  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:15:39.733989  7245 leader_election.cc:343] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.734061  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
W20260709 13:15:39.735935  7471 leader_election.cc:343] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.728379  7552 raft_consensus.cc:740] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
W20260709 13:15:39.738528  7468 leader_election.cc:343] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.725749  7550 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:39.730607  7556 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.739056  7468 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.741753  7556 raft_consensus.cc:2749] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.742439  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eb1ec31ba3340fb815cc11f5c116aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.743831  7558 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.744347  7558 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.744608  7552 consensus_queue.cc:260] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.746542  7558 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.747689  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.746506  7177 catalog_manager.cc:5697] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.718886  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 2eb1ec31ba3340fb815cc11f5c116aa0 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:39.749073  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc3465e298174cd9b6cdf4a7ea730ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.748924  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:39.738021  7580 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:39.749769  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eb1ec31ba3340fb815cc11f5c116aa0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:39.749981  7397 leader_election.cc:343] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:39.750993  7394 leader_election.cc:343] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.751305  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:39.751540  7394 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:39.752019  7505 raft_consensus.cc:2468] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:39.752653  7558 raft_consensus.cc:2749] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.753783  7394 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.754520  7554 tablet_bootstrap.cc:654] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.754931  7558 raft_consensus.cc:2804] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:39.755407  7558 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.755957  7558 raft_consensus.cc:3060] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.763046  7558 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.765993  7552 ts_tablet_manager.cc:1434] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:15:39.726686  7571 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:39.766322  7344 tablet_service.cc:1511] Processing CreateTablet for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:39.775245  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.770644  7357 raft_consensus.cc:2468] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:39.771883  7550 ts_tablet_manager.cc:1403] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.006s
I20260709 13:15:39.773849  7552 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:39.784461  7263 tablet_service.cc:1511] Processing CreateTablet for tablet a71872601e484b06882011698f8b3b52 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:39.785025  7551 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:39.785625  7551 ts_tablet_manager.cc:1403] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.128s	user 0.014s	sys 0.003s
I20260709 13:15:39.785928  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71872601e484b06882011698f8b3b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.785992  7248 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:39.788480  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:39.788720  7493 tablet_service.cc:1511] Processing CreateTablet for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:39.789122  7505 raft_consensus.cc:3060] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:39.790091  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b8f1e9ac0b246e5b46bd9d85336d5eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.791903  7571 raft_consensus.cc:2804] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:39.793932  7551 raft_consensus.cc:359] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.794718  7551 raft_consensus.cc:385] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.795019  7551 raft_consensus.cc:740] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.796311  7571 raft_consensus.cc:697] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:39.798012  7505 raft_consensus.cc:2468] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:39.804117  7394 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:39.804194  7551 consensus_queue.cc:260] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.797762  7550 raft_consensus.cc:359] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.808197  7550 raft_consensus.cc:385] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.808655  7550 raft_consensus.cc:740] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.809845  7550 consensus_queue.cc:260] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:39.799302  7571 consensus_queue.cc:237] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.773350  7356 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:39.802753  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:39.829213  7248 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:39.830895  7580 raft_consensus.cc:2804] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:39.831702  7580 raft_consensus.cc:697] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
W20260709 13:15:39.833670  7397 leader_election.cc:343] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.832829  7580 consensus_queue.cc:237] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:39.838876  7556 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.839685  7556 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:39.830679  7347 tablet_service.cc:1511] Processing CreateTablet for tablet a71872601e484b06882011698f8b3b52 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:39.844208  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:39.844904  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71872601e484b06882011698f8b3b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.845474  7556 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
W20260709 13:15:39.847299  7468 leader_election.cc:343] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.850515  7552 tablet_bootstrap.cc:654] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.816079  7558 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:39.852013  7558 raft_consensus.cc:493] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.852614  7558 raft_consensus.cc:515] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.856307  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:39.857788  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:39.859730  7471 leader_election.cc:343] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.860337  7471 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.885098  7596 raft_consensus.cc:2804] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:39.885696  7596 raft_consensus.cc:697] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:39.886576  7551 ts_tablet_manager.cc:1434] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.101s	user 0.005s	sys 0.012s
I20260709 13:15:39.886552  7596 consensus_queue.cc:237] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
W20260709 13:15:39.861861  7397 leader_election.cc:343] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.890674  7550 ts_tablet_manager.cc:1434] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.002s
I20260709 13:15:39.891206  7551 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:39.875468  7556 raft_consensus.cc:2749] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:39.884114  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 80089.4us
I20260709 13:15:39.922400  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 8af02e3c522a4048b817ec444a3ad5e6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:39.924115  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 8af02e3c522a4048b817ec444a3ad5e6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:39.917415  7550 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:39.930143  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af02e3c522a4048b817ec444a3ad5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.931896  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:39.926056  7558 leader_election.cc:290] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:39.941085  7414 tablet_service.cc:1511] Processing CreateTablet for tablet a71872601e484b06882011698f8b3b52 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:39.942513  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71872601e484b06882011698f8b3b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.930119  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af02e3c522a4048b817ec444a3ad5e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:39.934165  7394 leader_election.cc:343] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:39.952366  7347 tablet_service.cc:1511] Processing CreateTablet for tablet 8af02e3c522a4048b817ec444a3ad5e6 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:39.954243  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af02e3c522a4048b817ec444a3ad5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.948627  7394 leader_election.cc:304] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:39.957046  7551 tablet_bootstrap.cc:654] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.958315  7550 tablet_bootstrap.cc:654] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:39.967168  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad0292559534756b194428330c7c717 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:39.968513  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad0292559534756b194428330c7c717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.965770  7178 catalog_manager.cc:5697] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.969386  7554 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:39.969928  7554 ts_tablet_manager.cc:1403] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.294s	user 0.009s	sys 0.009s
I20260709 13:15:39.962206  7596 raft_consensus.cc:2749] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:39.981757  7493 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad0292559534756b194428330c7c717 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:39.981606  7554 raft_consensus.cc:359] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.982693  7554 raft_consensus.cc:385] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.983083  7554 raft_consensus.cc:740] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.983136  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad0292559534756b194428330c7c717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:39.984349  7554 consensus_queue.cc:260] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.986976  7554 ts_tablet_manager.cc:1434] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:15:39.988368  7554 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:39.993501  7552 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:39.994032  7552 ts_tablet_manager.cc:1403] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.221s	user 0.037s	sys 0.011s
I20260709 13:15:39.997059  7552 raft_consensus.cc:359] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.997823  7552 raft_consensus.cc:385] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:39.998114  7552 raft_consensus.cc:740] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:39.998468  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad0292559534756b194428330c7c717 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:39.998847  7552 consensus_queue.cc:260] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:39.999933  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad0292559534756b194428330c7c717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.000000  7580 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.008814  7551 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:40.008720  7580 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.008879  7486 tablet_service.cc:1511] Processing CreateTablet for tablet aa22447b4d024658b6470d416092c4cc (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:40.009609  7551 ts_tablet_manager.cc:1403] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.014s
I20260709 13:15:40.010495  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa22447b4d024658b6470d416092c4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.010963  7580 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.013000  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.001283  7552 ts_tablet_manager.cc:1434] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:40.001830  7344 tablet_service.cc:1511] Processing CreateTablet for tablet aa22447b4d024658b6470d416092c4cc (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:40.015002  7552 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.015210  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa22447b4d024658b6470d416092c4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.015653  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:40.016991  7245 leader_election.cc:343] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.017599  7551 raft_consensus.cc:359] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.018430  7551 raft_consensus.cc:385] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.004395  7554 tablet_bootstrap.cc:654] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.006214  7415 tablet_service.cc:1511] Processing CreateTablet for tablet aa22447b4d024658b6470d416092c4cc (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:40.021744  7415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa22447b4d024658b6470d416092c4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.003873  7596 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.027846  7596 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.029018  7430 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:40.029458  7552 tablet_bootstrap.cc:654] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.031452  7245 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:40.032907  7580 raft_consensus.cc:2804] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.033334  7580 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.033702  7580 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.035683  7550 tablet_bootstrap.cc:492] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.040783  7271 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5fcfd6d243e5940a37ab978b4f2b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:40.041868  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.042623  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5fcfd6d243e5940a37ab978b4f2b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:40.043565  7394 leader_election.cc:343] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.036288  7550 ts_tablet_manager.cc:1403] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.125s	user 0.021s	sys 0.001s
I20260709 13:15:40.048712  7596 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:40.018774  7551 raft_consensus.cc:740] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.048929  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:40.050004  7551 consensus_queue.cc:260] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.051033  7550 raft_consensus.cc:359] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.058482  7347 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5fcfd6d243e5940a37ab978b4f2b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:40.059191  7551 ts_tablet_manager.cc:1434] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:15:40.059562  7580 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.061091  7282 raft_consensus.cc:2393] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:40.062054  7580 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.063073  7394 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:40.063736  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.064438  7430 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.064024  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5fcfd6d243e5940a37ab978b4f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.066679  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
W20260709 13:15:40.068333  7245 leader_election.cc:343] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.080772  7414 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5fcfd6d243e5940a37ab978b4f2b (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:40.082172  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5fcfd6d243e5940a37ab978b4f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.087815  7580 consensus_queue.cc:1048] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:40.088368  7271 tablet_service.cc:1511] Processing CreateTablet for tablet ef5aa9b772e6445c83184f5551976222 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:40.089146  7346 tablet_service.cc:1511] Processing CreateTablet for tablet ef5aa9b772e6445c83184f5551976222 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:40.089730  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5aa9b772e6445c83184f5551976222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.090531  7346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5aa9b772e6445c83184f5551976222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.092051  7556 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.060371  7551 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:40.054534  7550 raft_consensus.cc:385] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.092592  7556 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.095065  7550 raft_consensus.cc:740] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.111574  7421 tablet_service.cc:1511] Processing CreateTablet for tablet 0f21194252b0411c9c67f78fcfec3140 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:40.113068  7421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f21194252b0411c9c67f78fcfec3140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.075924  7493 tablet_service.cc:1511] Processing CreateTablet for tablet ef5aa9b772e6445c83184f5551976222 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:40.099970  7430 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:40.114195  7177 catalog_manager.cc:5697] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.120012  7596 raft_consensus.cc:2749] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.120851  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 0f21194252b0411c9c67f78fcfec3140 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:40.122166  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f21194252b0411c9c67f78fcfec3140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.128427  7420 tablet_service.cc:1511] Processing CreateTablet for tablet 8142e76af31447acb3186673c2a9934a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:40.129750  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8142e76af31447acb3186673c2a9934a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.133956  7347 tablet_service.cc:1511] Processing CreateTablet for tablet 8142e76af31447acb3186673c2a9934a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:40.135174  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8142e76af31447acb3186673c2a9934a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.136554  7493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5aa9b772e6445c83184f5551976222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.137580  7554 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:40.138111  7554 ts_tablet_manager.cc:1403] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.150s	user 0.009s	sys 0.003s
I20260709 13:15:40.140494  7271 tablet_service.cc:1511] Processing CreateTablet for tablet 0f21194252b0411c9c67f78fcfec3140 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:40.141698  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f21194252b0411c9c67f78fcfec3140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.142549  7551 tablet_bootstrap.cc:654] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.142788  7556 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:40.146031  7580 raft_consensus.cc:493] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.146539  7580 raft_consensus.cc:515] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.146632  7554 raft_consensus.cc:359] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.125842  7550 consensus_queue.cc:260] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.148350  7245 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:40.148574  7580 leader_election.cc:290] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.144256  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.149322  7431 raft_consensus.cc:2468] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.150156  7571 raft_consensus.cc:2804] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.150686  7571 raft_consensus.cc:697] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:40.150792  7468 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:40.145042  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:40.151626  7571 consensus_queue.cc:237] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.157601  7556 raft_consensus.cc:2804] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.158317  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.158878  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.159538  7489 tablet_service.cc:1511] Processing CreateTablet for tablet 8142e76af31447acb3186673c2a9934a (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:40.158874  7556 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:40.160420  7245 leader_election.cc:343] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.160517  7556 raft_consensus.cc:3060] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.160940  7489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8142e76af31447acb3186673c2a9934a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:40.161240  7245 leader_election.cc:343] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.161715  7245 leader_election.cc:304] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:40.162503  7585 raft_consensus.cc:2749] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.164405  7177 catalog_manager.cc:5697] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.172623  7263 tablet_service.cc:1511] Processing CreateTablet for tablet 56bbbbedcee0446abe898050f5dd1578 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:40.173224  7344 tablet_service.cc:1511] Processing CreateTablet for tablet 56bbbbedcee0446abe898050f5dd1578 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:40.174168  7263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbbbedcee0446abe898050f5dd1578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.174525  7344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbbbedcee0446abe898050f5dd1578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.167446  7550 ts_tablet_manager.cc:1434] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.120s	user 0.008s	sys 0.000s
I20260709 13:15:40.170313  7552 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:40.147446  7554 raft_consensus.cc:385] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.183423  7554 raft_consensus.cc:740] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.196441  7554 consensus_queue.cc:260] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.198766  7554 ts_tablet_manager.cc:1434] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:15:40.184998  7552 ts_tablet_manager.cc:1403] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.170s	user 0.019s	sys 0.026s
I20260709 13:15:40.199924  7550 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.202119  7552 raft_consensus.cc:359] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.202848  7552 raft_consensus.cc:385] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.203155  7552 raft_consensus.cc:740] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.211160  7552 consensus_queue.cc:260] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.213686  7552 ts_tablet_manager.cc:1434] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 13:15:40.215093  7552 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.217572  7556 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:40.229921  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 78866.5us
W20260709 13:15:40.230963  7468 leader_election.cc:343] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.231267  7414 tablet_service.cc:1511] Processing CreateTablet for tablet 56bbbbedcee0446abe898050f5dd1578 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:40.232090  7551 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:40.232676  7551 ts_tablet_manager.cc:1403] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.173s	user 0.019s	sys 0.000s
I20260709 13:15:40.232992  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbbbedcee0446abe898050f5dd1578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.236485  7551 raft_consensus.cc:359] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.237259  7551 raft_consensus.cc:385] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.237576  7551 raft_consensus.cc:740] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.238312  7177 catalog_manager.cc:5697] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:40.241183  7245 consensus_peers.cc:597] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d -> Peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Couldn't send request to peer 49f7f0d3dec548bbb5538d2b1ddb4415. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.242575  7556 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:40.244292  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.244431  7551 consensus_queue.cc:260] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.245223  7431 raft_consensus.cc:3060] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.248114  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:40.250625  7271 tablet_service.cc:1511] Processing CreateTablet for tablet dc1db9e93f304836b7e9f3bc59c89600 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:40.252103  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc1db9e93f304836b7e9f3bc59c89600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.248133  7554 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:40.242612  7550 tablet_bootstrap.cc:654] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:40.263675  7468 leader_election.cc:343] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.265414  7571 consensus_queue.cc:1048] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:40.266755  7431 raft_consensus.cc:2468] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:40.268347  7468 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:40.269791  7556 raft_consensus.cc:2804] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.270273  7556 raft_consensus.cc:697] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:40.271323  7596 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.271159  7556 consensus_queue.cc:237] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.271812  7596 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.273028  7564 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.273532  7564 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.273969  7596 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.276692  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.278163  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:40.278883  7397 leader_election.cc:343] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.276731  7564 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
W20260709 13:15:40.279773  7394 leader_election.cc:343] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.280591  7394 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:40.304388  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.305172  7431 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:40.305977  7552 tablet_bootstrap.cc:654] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.272226  7486 tablet_service.cc:1511] Processing CreateTablet for tablet dc1db9e93f304836b7e9f3bc59c89600 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:40.308975  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc1db9e93f304836b7e9f3bc59c89600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.309834  7178 catalog_manager.cc:5697] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.297336  7596 raft_consensus.cc:2749] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.333364  7550 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.334277  7550 ts_tablet_manager.cc:1403] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.135s	user 0.014s	sys 0.007s
I20260709 13:15:40.337972  7347 tablet_service.cc:1511] Processing CreateTablet for tablet dc1db9e93f304836b7e9f3bc59c89600 (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:40.339432  7554 tablet_bootstrap.cc:654] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.339505  7347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc1db9e93f304836b7e9f3bc59c89600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.338125  7550 raft_consensus.cc:359] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.358325  7420 tablet_service.cc:1511] Processing CreateTablet for tablet f035dcc1e4cc4e6fae948203b417e7cd (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:40.359704  7420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f035dcc1e4cc4e6fae948203b417e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.364661  7610 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.365269  7610 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.368897  7610 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.368780  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.370000  7431 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.370253  7552 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:40.370846  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.371738  7468 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:40.368739  7486 tablet_service.cc:1511] Processing CreateTablet for tablet f035dcc1e4cc4e6fae948203b417e7cd (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:40.375638  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f035dcc1e4cc4e6fae948203b417e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.375919  7320 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.376796  7271 tablet_service.cc:1511] Processing CreateTablet for tablet f035dcc1e4cc4e6fae948203b417e7cd (DEFAULT_TABLE table=test-workload [id=02746edca4a044ae944ba99e96ea1c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:40.382016  7271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f035dcc1e4cc4e6fae948203b417e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:40.354882  7550 raft_consensus.cc:385] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.389375  7551 ts_tablet_manager.cc:1434] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.156s	user 0.009s	sys 0.000s
I20260709 13:15:40.389199  7431 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.390743  7551 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:40.400393  7571 consensus_queue.cc:1048] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:40.380683  7356 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.376489  7552 ts_tablet_manager.cc:1403] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.162s	user 0.020s	sys 0.006s
I20260709 13:15:40.403728  7571 consensus_queue.cc:1048] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:40.405616  7552 raft_consensus.cc:359] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.406404  7552 raft_consensus.cc:385] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.411542  7564 raft_consensus.cc:2804] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.412001  7564 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.412348  7564 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.407087  7550 raft_consensus.cc:740] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.416929  7550 consensus_queue.cc:260] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.419524  7550 ts_tablet_manager.cc:1434] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.085s	user 0.006s	sys 0.001s
I20260709 13:15:40.422986  7550 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.410985  7610 raft_consensus.cc:2804] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.424679  7610 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.425060  7610 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.429078  7554 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:40.432127  7554 ts_tablet_manager.cc:1403] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.001s
I20260709 13:15:40.435067  7554 raft_consensus.cc:359] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.406710  7552 raft_consensus.cc:740] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.435851  7554 raft_consensus.cc:385] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.438222  7554 raft_consensus.cc:740] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.439155  7554 consensus_queue.cc:260] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.436276  7564 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:15:40.447395  7245 consensus_peers.cc:597] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.447983  7550 tablet_bootstrap.cc:654] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.457084  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.459004  7551 tablet_bootstrap.cc:654] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.449208  7552 consensus_queue.cc:260] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.462064  7554 ts_tablet_manager.cc:1434] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 13:15:40.463232  7554 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:40.469373  7554 tablet_bootstrap.cc:654] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.470479  7552 ts_tablet_manager.cc:1434] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.001s
I20260709 13:15:40.472234  7610 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.472769  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.474299  7564 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.476581  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:40.477877  7430 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.486964  7505 raft_consensus.cc:2393] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:40.489109  7610 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.490118  7554 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:40.490698  7554 ts_tablet_manager.cc:1403] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.009s
I20260709 13:15:40.490904  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.484782  7571 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.493628  7571 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.494848  7554 raft_consensus.cc:359] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.495553  7554 raft_consensus.cc:385] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.495718  7571 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:40.495867  7554 raft_consensus.cc:740] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.496919  7554 consensus_queue.cc:260] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.498414  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.492203  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.499907  7357 raft_consensus.cc:2393] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:40.502947  7471 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:40.503518  7622 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.503839  7610 raft_consensus.cc:2749] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.504074  7622 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.492668  7552 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.509783  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.510514  7357 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:40.510695  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.511521  7431 raft_consensus.cc:2393] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b2b311085b64300925884dec9a28c7d in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
W20260709 13:15:40.514469  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:40.514770  7552 tablet_bootstrap.cc:654] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.516347  7248 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.517334  7580 raft_consensus.cc:2804] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.517701  7580 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.518056  7580 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.500713  7554 ts_tablet_manager.cc:1434] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:15:40.523473  7596 consensus_queue.cc:1048] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:40.506417  7622 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.520792  7554 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:40.525242  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.525974  7505 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:40.529129  7580 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.531327  7245 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:40.535548  7554 tablet_bootstrap.cc:654] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:40.535430  7397 consensus_peers.cc:597] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.538452  7622 raft_consensus.cc:3060] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.540155  7550 tablet_bootstrap.cc:492] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.540839  7550 ts_tablet_manager.cc:1403] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.018s
I20260709 13:15:40.544585  7430 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:40.548336  7551 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:40.548893  7551 ts_tablet_manager.cc:1403] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.158s	user 0.017s	sys 0.016s
I20260709 13:15:40.549106  7550 raft_consensus.cc:359] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.549839  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:40.550135  7550 raft_consensus.cc:385] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.550500  7504 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.550544  7550 raft_consensus.cc:740] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.551920  7551 raft_consensus.cc:359] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.552290  7320 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:40.553009  7550 consensus_queue.cc:260] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.554123  7622 raft_consensus.cc:2749] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.554842  7624 raft_consensus.cc:2804] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.553306  7551 raft_consensus.cc:385] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.555492  7551 raft_consensus.cc:740] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.555380  7624 raft_consensus.cc:697] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:40.556243  7551 consensus_queue.cc:260] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.556734  7550 ts_tablet_manager.cc:1434] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:15:40.556883  7624 consensus_queue.cc:237] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.558235  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.558717  7551 ts_tablet_manager.cc:1434] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:40.558816  7357 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.560077  7551 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:40.565058  7550 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.570863  7551 tablet_bootstrap.cc:654] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:40.575863  7245 consensus_peers.cc:597] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d -> Peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Couldn't send request to peer 49f7f0d3dec548bbb5538d2b1ddb4415. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.576043  7357 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:40.577302  7248 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.578166  7622 raft_consensus.cc:2804] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.579171  7622 raft_consensus.cc:697] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:40.580050  7622 consensus_queue.cc:237] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.566846  7580 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.600273  7552 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:40.600973  7552 ts_tablet_manager.cc:1403] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.007s
I20260709 13:15:40.604074  7550 tablet_bootstrap.cc:654] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.609386  7552 raft_consensus.cc:359] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.611117  7552 raft_consensus.cc:385] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.611559  7552 raft_consensus.cc:740] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.614076  7552 consensus_queue.cc:260] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.612919  7504 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:40.626883  7551 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:40.627688  7551 ts_tablet_manager.cc:1403] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.000s
I20260709 13:15:40.628147  7178 catalog_manager.cc:5697] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.631654  7551 raft_consensus.cc:359] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.632486  7551 raft_consensus.cc:385] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.632788  7551 raft_consensus.cc:740] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.633653  7551 consensus_queue.cc:260] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.637171  7552 ts_tablet_manager.cc:1434] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:15:40.638787  7552 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.640918  7177 catalog_manager.cc:5697] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.645300  7550 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.645844  7550 ts_tablet_manager.cc:1403] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.081s	user 0.017s	sys 0.006s
I20260709 13:15:40.645944  7554 tablet_bootstrap.cc:492] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:40.646502  7554 ts_tablet_manager.cc:1403] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.050s
I20260709 13:15:40.648437  7550 raft_consensus.cc:359] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.648959  7554 raft_consensus.cc:359] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.649499  7550 raft_consensus.cc:385] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.649672  7554 raft_consensus.cc:385] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.649907  7550 raft_consensus.cc:740] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.650038  7554 raft_consensus.cc:740] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.651381  7552 tablet_bootstrap.cc:654] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.650906  7550 consensus_queue.cc:260] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.653422  7550 ts_tablet_manager.cc:1434] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:15:40.651006  7554 consensus_queue.cc:260] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.655076  7550 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.656251  7554 ts_tablet_manager.cc:1434] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:40.657498  7554 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:40.657717  7564 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.658221  7564 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.660403  7564 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.663033  7550 tablet_bootstrap.cc:654] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.663648  7554 tablet_bootstrap.cc:654] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.663897  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:40.664737  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:40.665342  7320 leader_election.cc:343] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:40.666414  7320 leader_election.cc:343] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.666860  7320 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:40.667964  7624 raft_consensus.cc:2749] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.668319  7552 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:40.668802  7552 ts_tablet_manager.cc:1403] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:15:40.671911  7552 raft_consensus.cc:359] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.672545  7552 raft_consensus.cc:385] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.672420  7610 raft_consensus.cc:493] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.673053  7552 raft_consensus.cc:740] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.673385  7610 raft_consensus.cc:515] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.673873  7552 consensus_queue.cc:260] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.678539  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.679394  7357 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.679807  7624 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.679518  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.680322  7624 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.680572  7430 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.682461  7624 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.682571  7468 leader_election.cc:304] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:40.683827  7556 raft_consensus.cc:2804] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.684202  7556 raft_consensus.cc:493] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.684314  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.684571  7556 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.689859  7610 leader_election.cc:290] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.692588  7552 ts_tablet_manager.cc:1434] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:15:40.688711  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.694877  7320 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:40.696414  7624 raft_consensus.cc:2749] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:40.699790  7551 ts_tablet_manager.cc:1434] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.000s
I20260709 13:15:40.700917  7551 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:40.706322  7552 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.711294  7580 consensus_queue.cc:1048] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:40.715005  7552 tablet_bootstrap.cc:654] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.706403  7551 tablet_bootstrap.cc:654] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.718151  7550 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.718677  7550 ts_tablet_manager.cc:1403] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.029s
I20260709 13:15:40.721786  7550 raft_consensus.cc:359] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.722695  7550 raft_consensus.cc:385] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.723007  7550 raft_consensus.cc:740] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.723897  7550 consensus_queue.cc:260] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:40.736745  7610 consensus_queue.cc:1048] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:40.737347  7550 ts_tablet_manager.cc:1434] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:15:40.738556  7551 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:40.739078  7551 ts_tablet_manager.cc:1403] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 13:15:40.739078  7550 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.742014  7551 raft_consensus.cc:359] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.742763  7551 raft_consensus.cc:385] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.743049  7551 raft_consensus.cc:740] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.745510  7552 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:40.746016  7552 ts_tablet_manager.cc:1403] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.009s
I20260709 13:15:40.746047  7550 tablet_bootstrap.cc:654] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.749579  7552 raft_consensus.cc:359] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.750542  7552 raft_consensus.cc:385] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.750905  7552 raft_consensus.cc:740] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.744031  7551 consensus_queue.cc:260] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.752933  7635 raft_consensus.cc:493] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.753538  7635 raft_consensus.cc:515] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.758917  7610 raft_consensus.cc:493] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.759344  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.759769  7610 raft_consensus.cc:515] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:15:40.761404  7397 leader_election.cc:343] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.763698  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:40.764012  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:40.764696  7282 raft_consensus.cc:2468] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:40.765012  7281 raft_consensus.cc:2468] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.766794  7394 leader_election.cc:304] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:40.766923  7468 leader_election.cc:304] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:40.768280  7629 raft_consensus.cc:2804] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.768769  7629 raft_consensus.cc:493] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.769223  7629 raft_consensus.cc:3060] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.774672  7610 leader_election.cc:290] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.775493  7610 raft_consensus.cc:2804] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.775983  7610 raft_consensus.cc:493] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.776386  7610 raft_consensus.cc:3060] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.779877  7635 leader_election.cc:290] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.784332  7556 raft_consensus.cc:515] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.788028  7552 consensus_queue.cc:260] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.788419  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.789470  7357 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.794100  7505 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.794341  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:40.795059  7430 raft_consensus.cc:2468] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:40.800288  7357 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:40.810566  7471 leader_election.cc:304] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.801229  7629 raft_consensus.cc:515] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.820628  7551 ts_tablet_manager.cc:1434] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.006s
I20260709 13:15:40.822238  7551 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:40.811810  7648 raft_consensus.cc:2804] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.825750  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:40.826637  7281 raft_consensus.cc:3060] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.828915  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.823164  7650 consensus_queue.cc:1048] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:40.809034  7552 ts_tablet_manager.cc:1434] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.002s
W20260709 13:15:40.830639  7397 leader_election.cc:343] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.831403  7629 leader_election.cc:290] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.833923  7552 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:40.840126  7550 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.842587  7550 ts_tablet_manager.cc:1403] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.104s	user 0.048s	sys 0.038s
I20260709 13:15:40.845419  7550 raft_consensus.cc:359] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.846127  7550 raft_consensus.cc:385] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.846413  7550 raft_consensus.cc:740] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.847093  7550 consensus_queue.cc:260] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:15:40.849218  7646 log.cc:927] Time spent T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:15:40.849375  7550 ts_tablet_manager.cc:1434] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:40.850488  7550 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.850620  7551 tablet_bootstrap.cc:654] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.842092  7552 tablet_bootstrap.cc:654] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.866227  7550 tablet_bootstrap.cc:654] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.866415  7585 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.866992  7585 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.869117  7585 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.870777  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:40.872884  7248 leader_election.cc:343] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.874616  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:40.875311  7505 raft_consensus.cc:2468] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:40.877296  7245 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:40.878543  7585 raft_consensus.cc:2804] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.878965  7585 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.879383  7585 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.886194  7550 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.886963  7550 ts_tablet_manager.cc:1403] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.000s
I20260709 13:15:40.890444  7550 raft_consensus.cc:359] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.895689  7556 leader_election.cc:290] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.896904  7585 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.891388  7550 raft_consensus.cc:385] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.899910  7550 raft_consensus.cc:740] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.901242  7550 consensus_queue.cc:260] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.902870  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:40.903477  7505 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.904776  7281 raft_consensus.cc:2468] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:40.906121  7394 leader_election.cc:304] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:40.907249  7629 raft_consensus.cc:2804] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.907297  7550 ts_tablet_manager.cc:1434] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.004s
I20260709 13:15:40.907867  7629 raft_consensus.cc:697] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:40.908761  7629 consensus_queue.cc:237] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.915001  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.915280  7648 raft_consensus.cc:697] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:40.916244  7580 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.908741  7550 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.916795  7580 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.918028  7505 raft_consensus.cc:2468] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:40.919130  7580 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.920590  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:40.921207  7357 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:40.922595  7248 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.915685  7431 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.923527  7580 raft_consensus.cc:2804] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:40.923974  7580 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.924371  7580 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.932552  7245 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:40.927039  7550 tablet_bootstrap.cc:654] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:40.933665  7571 raft_consensus.cc:2804] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.934497  7571 raft_consensus.cc:697] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:40.934955  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.935353  7571 consensus_queue.cc:237] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:40.936547  7248 leader_election.cc:343] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.916455  7648 consensus_queue.cc:237] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:40.937450  7585 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:40.927892  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:40.940958  7245 leader_election.cc:343] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.941182  7580 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.947154  7550 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:40.950430  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:40.950786  7550 ts_tablet_manager.cc:1403] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.003s
I20260709 13:15:40.951086  7357 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:40.955911  7550 raft_consensus.cc:359] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.957034  7550 raft_consensus.cc:385] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:40.957465  7550 raft_consensus.cc:740] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:40.958307  7550 consensus_queue.cc:260] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:40.959707  7357 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:40.961023  7248 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:40.961970  7571 raft_consensus.cc:2804] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:40.962487  7571 raft_consensus.cc:697] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:40.963572  7571 consensus_queue.cc:237] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:40.967943  7580 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:40.969286  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:40.968245  7550 ts_tablet_manager.cc:1434] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
W20260709 13:15:40.970924  7245 leader_election.cc:343] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:40.971439  7550 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:40.975023  7554 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:40.975504  7554 ts_tablet_manager.cc:1403] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.318s	user 0.065s	sys 0.075s
I20260709 13:15:40.979008  7554 raft_consensus.cc:359] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.000172  7554 raft_consensus.cc:385] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.000553  7554 raft_consensus.cc:740] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.000613  7550 tablet_bootstrap.cc:654] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.001781  7554 consensus_queue.cc:260] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.006716  7429 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.008812  7357 raft_consensus.cc:3060] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.011646  7431 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.017760  7505 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.018730  7552 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.019177  7552 ts_tablet_manager.cc:1403] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.185s	user 0.027s	sys 0.049s
I20260709 13:15:41.019310  7178 catalog_manager.cc:5697] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.021523  7552 raft_consensus.cc:359] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.022105  7552 raft_consensus.cc:385] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.022377  7552 raft_consensus.cc:740] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.025305  7550 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.025995  7551 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:41.025902  7580 consensus_queue.cc:1048] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:41.026662  7551 ts_tablet_manager.cc:1403] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.205s	user 0.055s	sys 0.087s
I20260709 13:15:41.027434  7177 catalog_manager.cc:5697] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.031512  7550 ts_tablet_manager.cc:1403] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.004s
I20260709 13:15:41.034966  7550 raft_consensus.cc:359] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.035724  7550 raft_consensus.cc:385] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.036039  7550 raft_consensus.cc:740] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.036911  7552 consensus_queue.cc:260] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.039299  7552 ts_tablet_manager.cc:1434] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:15:41.040536  7610 raft_consensus.cc:515] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.043304  7552 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.044375  7550 consensus_queue.cc:260] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.050133  7624 consensus_queue.cc:1048] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.050650  7550 ts_tablet_manager.cc:1434] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.006s
I20260709 13:15:41.052116  7550 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.053858  7610 leader_election.cc:290] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.045315  7554 ts_tablet_manager.cc:1434] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:15:41.057262  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:41.059857  7428 raft_consensus.cc:3060] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.059403  7177 catalog_manager.cc:5697] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.060736  7554 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:41.061493  7177 catalog_manager.cc:5697] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.046103  7551 raft_consensus.cc:359] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.063803  7551 raft_consensus.cc:385] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.064162  7551 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.065089  7551 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.066975  7550 tablet_bootstrap.cc:654] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.067631  7551 ts_tablet_manager.cc:1434] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 13:15:41.068692  7551 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.073680  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:41.057868  7552 tablet_bootstrap.cc:654] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.078485  7281 raft_consensus.cc:3060] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.082888  7551 tablet_bootstrap.cc:654] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.096072  7428 raft_consensus.cc:2468] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.102290  7610 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.118988  7468 leader_election.cc:304] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.122115  7610 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.123773  7648 raft_consensus.cc:2804] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.124610  7648 raft_consensus.cc:697] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:41.125221  7610 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.130690  7554 tablet_bootstrap.cc:654] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.131879  7650 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.130348  7564 consensus_queue.cc:1048] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.132624  7650 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.125808  7648 consensus_queue.cc:237] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.141280  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.135449  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.123751  7281 raft_consensus.cc:2468] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
W20260709 13:15:41.152976  7468 leader_election.cc:343] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:41.173666  7471 leader_election.cc:343] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.177660  7650 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:41.180177  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:41.182356  7471 leader_election.cc:343] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.183017  7471 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.154727  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:41.186081  7468 leader_election.cc:343] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.186581  7468 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.187785  7610 raft_consensus.cc:2749] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.193588  7551 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:41.194409  7551 ts_tablet_manager.cc:1403] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.126s	user 0.009s	sys 0.008s
I20260709 13:15:41.192684  7650 raft_consensus.cc:2749] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.199026  7552 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.199566  7552 ts_tablet_manager.cc:1403] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.015s
I20260709 13:15:41.202239  7550 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.202854  7550 ts_tablet_manager.cc:1403] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.151s	user 0.026s	sys 0.013s
I20260709 13:15:41.203850  7552 raft_consensus.cc:359] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.204443  7552 raft_consensus.cc:385] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.204708  7552 raft_consensus.cc:740] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.208422  7551 raft_consensus.cc:359] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.209180  7551 raft_consensus.cc:385] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.209412  7551 raft_consensus.cc:740] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.210273  7551 consensus_queue.cc:260] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.213089  7585 consensus_queue.cc:1048] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.217540  7551 ts_tablet_manager.cc:1434] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:15:41.218621  7551 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.221189  7650 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.220847  7550 raft_consensus.cc:359] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.221776  7550 raft_consensus.cc:385] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.222275  7550 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.221745  7650 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.222944  7550 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.224148  7552 consensus_queue.cc:260] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.225226  7550 ts_tablet_manager.cc:1434] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 13:15:41.226102  7552 ts_tablet_manager.cc:1434] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:15:41.227770  7552 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.230895  7650 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:41.226219  7550 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.226300  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.232362  7281 raft_consensus.cc:2468] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:41.233661  7468 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:41.234539  7650 raft_consensus.cc:2804] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.234933  7650 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.235275  7650 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.237255  7552 tablet_bootstrap.cc:654] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.240458  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:41.242054  7471 leader_election.cc:343] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.242311  7550 tablet_bootstrap.cc:654] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.245733  7554 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:41.246050  7580 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.246234  7554 ts_tablet_manager.cc:1403] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.186s	user 0.006s	sys 0.034s
I20260709 13:15:41.246628  7580 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.248425  7552 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.249842  7552 ts_tablet_manager.cc:1403] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.005s
I20260709 13:15:41.251941  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:41.253371  7248 leader_election.cc:343] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.253738  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.255390  7551 tablet_bootstrap.cc:654] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:41.256141  7245 leader_election.cc:343] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.256803  7245 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.258461  7585 raft_consensus.cc:2749] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.258592  7176 catalog_manager.cc:5697] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.260762  7650 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.249101  7580 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.248971  7554 raft_consensus.cc:359] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.263512  7554 raft_consensus.cc:385] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.263847  7554 raft_consensus.cc:740] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.263950  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:41.265266  7281 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.264971  7554 consensus_queue.cc:260] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.267040  7552 raft_consensus.cc:359] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.267575  7554 ts_tablet_manager.cc:1434] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:15:41.267817  7552 raft_consensus.cc:385] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.268221  7552 raft_consensus.cc:740] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.269019  7554 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:41.268946  7552 consensus_queue.cc:260] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.271147  7552 ts_tablet_manager.cc:1434] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:15:41.272190  7552 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.275588  7554 tablet_bootstrap.cc:654] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.276821  7281 raft_consensus.cc:2468] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.278071  7552 tablet_bootstrap.cc:654] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.278085  7468 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:41.279294  7648 raft_consensus.cc:2804] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.279716  7650 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:41.279803  7648 raft_consensus.cc:697] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:41.280786  7648 consensus_queue.cc:237] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.284989  7550 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.285532  7550 ts_tablet_manager.cc:1403] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.001s
I20260709 13:15:41.286863  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:15:41.288342  7471 leader_election.cc:343] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.288950  7550 raft_consensus.cc:359] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.289671  7550 raft_consensus.cc:385] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.289925  7550 raft_consensus.cc:740] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.290786  7550 consensus_queue.cc:260] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.293431  7550 ts_tablet_manager.cc:1434] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:41.294785  7550 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.295755  7551 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:41.296257  7551 ts_tablet_manager.cc:1403] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 13:15:41.299175  7551 raft_consensus.cc:359] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.299840  7551 raft_consensus.cc:385] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.300114  7551 raft_consensus.cc:740] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.300719  7551 consensus_queue.cc:260] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.302709  7551 ts_tablet_manager.cc:1434] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:41.293581  7552 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.304070  7552 ts_tablet_manager.cc:1403] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.004s
I20260709 13:15:41.306875  7552 raft_consensus.cc:359] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.307514  7552 raft_consensus.cc:385] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.307797  7552 raft_consensus.cc:740] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.308439  7552 consensus_queue.cc:260] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.309199  7550 tablet_bootstrap.cc:654] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.309401  7180 catalog_manager.cc:5697] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.310572  7552 ts_tablet_manager.cc:1434] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:41.311820  7552 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.317094  7554 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:41.317620  7554 ts_tablet_manager.cc:1403] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 13:15:41.318805  7551 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.320255  7554 raft_consensus.cc:359] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.321213  7554 raft_consensus.cc:385] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.321549  7554 raft_consensus.cc:740] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.322400  7648 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.322391  7554 consensus_queue.cc:260] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.323094  7648 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.329412  7648 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.325053  7551 tablet_bootstrap.cc:654] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.330130  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.330495  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:41.331485  7471 leader_election.cc:343] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:41.331985  7468 leader_election.cc:343] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.332126  7356 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.321902  7550 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.329483  7554 ts_tablet_manager.cc:1434] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:15:41.333114  7550 ts_tablet_manager.cc:1403] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.003s
I20260709 13:15:41.325361  7552 tablet_bootstrap.cc:654] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.333647  7554 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:41.332465  7468 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.334764  7648 raft_consensus.cc:2749] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.337287  7674 raft_consensus.cc:493] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.339084  7674 raft_consensus.cc:515] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.343149  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:41.343887  7505 raft_consensus.cc:2468] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:41.345714  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.347399  7320 leader_election.cc:304] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:41.348136  7554 tablet_bootstrap.cc:654] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.348346  7564 raft_consensus.cc:2804] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.349248  7564 raft_consensus.cc:493] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.349753  7564 raft_consensus.cc:3060] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.341351  7674 leader_election.cc:290] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:41.356968  7571 consensus_queue.cc:1048] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:15:41.359045  7320 leader_election.cc:343] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:41.372558  7245 consensus_peers.cc:597] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:41.373112  7564 raft_consensus.cc:515] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.374606  7580 consensus_queue.cc:1048] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.386798  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:41.387558  7648 consensus_queue.cc:1048] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:41.387722  7505 raft_consensus.cc:3060] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.391662  7355 raft_consensus.cc:3060] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.343633  7550 raft_consensus.cc:359] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.400733  7550 raft_consensus.cc:385] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.401636  7550 raft_consensus.cc:740] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.401209  7564 leader_election.cc:290] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:41.402990  7550 consensus_queue.cc:260] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.406479  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
W20260709 13:15:41.416381  7320 leader_election.cc:343] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.406991  7505 raft_consensus.cc:2468] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:41.418650  7320 leader_election.cc:304] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:41.418697  7629 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.419796  7629 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.421168  7564 raft_consensus.cc:2804] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.422137  7564 raft_consensus.cc:697] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:41.423684  7564 consensus_queue.cc:237] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.424122  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.427218  7550 ts_tablet_manager.cc:1434] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.000s
I20260709 13:15:41.429209  7551 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
W20260709 13:15:41.429373  7397 leader_election.cc:343] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.429709  7551 ts_tablet_manager.cc:1403] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.126s	user 0.041s	sys 0.044s
I20260709 13:15:41.430611  7629 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.432860  7551 raft_consensus.cc:359] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.433651  7551 raft_consensus.cc:385] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.433936  7551 raft_consensus.cc:740] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.434608  7551 consensus_queue.cc:260] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.439853  7629 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.441195  7629 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.444415  7552 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.445052  7552 ts_tablet_manager.cc:1403] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.134s	user 0.018s	sys 0.012s
I20260709 13:15:41.445555  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.446376  7281 raft_consensus.cc:2468] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:41.447781  7394 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.448190  7552 raft_consensus.cc:359] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.448982  7552 raft_consensus.cc:385] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.449047  7580 raft_consensus.cc:493] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.449306  7552 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.449633  7580 raft_consensus.cc:515] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.450139  7552 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.451865  7580 leader_election.cc:290] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.466984  7691 raft_consensus.cc:2804] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.468732  7550 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.468819  7691 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.469393  7691 raft_consensus.cc:3060] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.469679  7650 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.470247  7650 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.453584  7552 ts_tablet_manager.cc:1434] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:41.472767  7650 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:41.473723  7552 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.452706  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.460570  7551 ts_tablet_manager.cc:1434] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:15:41.476348  7551 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.478078  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.478904  7281 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:41.458639  7629 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:41.482685  7552 tablet_bootstrap.cc:654] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.460382  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.466390  7571 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.484275  7571 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.486423  7571 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.468218  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.487171  7551 tablet_bootstrap.cc:654] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.483814  7550 tablet_bootstrap.cc:654] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.489306  7610 consensus_queue.cc:1048] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:41.494311  7245 leader_election.cc:343] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.496958  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:41.497874  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.498237  7504 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:41.498615  7429 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:41.498905  7552 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.499490  7552 ts_tablet_manager.cc:1403] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.003s
I20260709 13:15:41.500499  7571 consensus_queue.cc:1048] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.502764  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.502656  7552 raft_consensus.cc:359] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.503501  7552 raft_consensus.cc:385] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.503544  7428 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:41.503959  7552 raft_consensus.cc:740] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.504791  7552 consensus_queue.cc:260] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.507218  7552 ts_tablet_manager.cc:1434] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:41.511128  7468 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:41.512944  7648 raft_consensus.cc:2804] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.513525  7648 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.513397  7691 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.514303  7648 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.516886  7552 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
W20260709 13:15:41.520889  7248 leader_election.cc:343] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.521798  7248 leader_election.cc:304] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.522710  7571 raft_consensus.cc:2749] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:41.458338  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:41.562711  7691 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:41.571307  7550 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.571905  7550 ts_tablet_manager.cc:1403] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.103s	user 0.024s	sys 0.008s
I20260709 13:15:41.575583  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:41.576692  7282 raft_consensus.cc:3060] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.576280  7551 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:41.578296  7551 ts_tablet_manager.cc:1403] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.000s
I20260709 13:15:41.585150  7550 raft_consensus.cc:359] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.585945  7550 raft_consensus.cc:385] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.586230  7550 raft_consensus.cc:740] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.586966  7550 consensus_queue.cc:260] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.591068  7552 tablet_bootstrap.cc:654] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.593245  7650 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.594280  7551 raft_consensus.cc:359] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.552359  7505 raft_consensus.cc:2468] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:41.599340  7551 raft_consensus.cc:385] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.609076  7551 raft_consensus.cc:740] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.610355  7551 consensus_queue.cc:260] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.599530  7550 ts_tablet_manager.cc:1434] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
W20260709 13:15:41.600461  7397 leader_election.cc:343] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.604456  7650 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.607198  7282 raft_consensus.cc:2468] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:41.614235  7550 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.559558  7245 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.629870  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:41.630249  7650 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:15:41.631783  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 74923.1us
W20260709 13:15:41.632588  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:41.633220  7622 raft_consensus.cc:2804] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.633816  7622 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.634068  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:41.634291  7622 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:41.634326  7471 leader_election.cc:343] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.638023  7394 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.640079  7691 raft_consensus.cc:2804] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.640522  7691 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.640880  7691 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.640919  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.641757  7428 raft_consensus.cc:2468] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:41.642796  7394 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.643836  7692 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.644019  7468 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.644325  7692 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.647367  7551 ts_tablet_manager.cc:1434] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 13:15:41.649545  7629 raft_consensus.cc:2804] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.652032  7692 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.651336  7551 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.661224  7550 tablet_bootstrap.cc:654] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:41.661325  7397 leader_election.cc:343] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.664213  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:41.665673  7397 leader_election.cc:343] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.666663  7691 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.649047  7650 raft_consensus.cc:2804] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.655660  7648 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.677872  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:41.678138  7610 consensus_queue.cc:1048] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:41.687647  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:41.679800  7650 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.683532  7692 consensus_queue.cc:1048] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.689143  7551 tablet_bootstrap.cc:654] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.679674  7629 raft_consensus.cc:697] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:41.688514  7648 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:41.691812  7610 consensus_queue.cc:1048] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.690166  7650 raft_consensus.cc:3060] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.694188  7674 raft_consensus.cc:493] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.694870  7691 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.693070  7503 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:41.696226  7281 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.696950  7692 consensus_queue.cc:1048] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.697288  7554 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:41.706075  7554 ts_tablet_manager.cc:1403] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.373s	user 0.055s	sys 0.117s
I20260709 13:15:41.703485  7552 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:41.707283  7552 ts_tablet_manager.cc:1403] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.191s	user 0.019s	sys 0.020s
I20260709 13:15:41.712791  7554 raft_consensus.cc:359] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.713819  7554 raft_consensus.cc:385] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.714195  7554 raft_consensus.cc:740] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.701829  7674 raft_consensus.cc:515] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.715335  7554 consensus_queue.cc:260] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.725049  7550 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.725625  7550 ts_tablet_manager.cc:1403] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.026s
I20260709 13:15:41.725991  7648 consensus_queue.cc:1048] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:41.726423  7552 raft_consensus.cc:359] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.726742  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.727123  7552 raft_consensus.cc:385] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.727497  7552 raft_consensus.cc:740] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.700306  7622 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.731451  7554 ts_tablet_manager.cc:1434] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:15:41.725098  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.741282  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:41.741667  7554 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:41.741935  7429 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.742761  7320 leader_election.cc:304] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:41.733457  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:41.744486  7428 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.729280  7674 leader_election.cc:290] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.735486  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:41.728775  7550 raft_consensus.cc:359] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.706966  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:41.729897  7552 consensus_queue.cc:260] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.737591  7180 catalog_manager.cc:5697] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.747308  7550 raft_consensus.cc:385] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.750092  7503 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.752151  7554 tablet_bootstrap.cc:654] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:41.759824  7471 consensus_peers.cc:597] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:41.761169  7650 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.764423  7394 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.771782  7550 raft_consensus.cc:740] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
W20260709 13:15:41.772838  7397 leader_election.cc:343] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.777021  7624 raft_consensus.cc:2749] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.799028  7552 ts_tablet_manager.cc:1434] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.000s
I20260709 13:15:41.800551  7552 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:41.746992  7629 consensus_queue.cc:237] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.808370  7428 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:41.810097  7245 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.811267  7585 raft_consensus.cc:2804] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.812084  7585 raft_consensus.cc:697] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:41.813169  7585 consensus_queue.cc:237] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.755021  7429 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.816892  7503 raft_consensus.cc:2468] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:41.838542  7550 consensus_queue.cc:260] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.843592  7550 ts_tablet_manager.cc:1434] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.118s	user 0.010s	sys 0.011s
I20260709 13:15:41.846884  7550 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.838406  7691 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:15:41.841658  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 77367us
I20260709 13:15:41.842658  7281 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.848610  7691 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.845026  7622 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.855883  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:41.856565  7503 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.857847  7650 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.859899  7468 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:41.860909  7650 raft_consensus.cc:2804] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.848805  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:15:41.864893  7471 leader_election.cc:343] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.862000  7650 raft_consensus.cc:697] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:41.871543  7674 raft_consensus.cc:493] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.853250  7691 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.875358  7550 tablet_bootstrap.cc:654] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.875562  7394 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.893759  7691 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.894436  7700 raft_consensus.cc:2804] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.895264  7700 raft_consensus.cc:697] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:41.896452  7700 consensus_queue.cc:237] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.897640  7691 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:41.879493  7674 raft_consensus.cc:515] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.900728  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:41.902235  7674 leader_election.cc:290] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:15:41.902274  7397 leader_election.cc:343] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.904466  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:41.908552  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:41.909075  7551 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:41.909222  7281 raft_consensus.cc:2468] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:41.909756  7551 ts_tablet_manager.cc:1403] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.262s	user 0.029s	sys 0.001s
I20260709 13:15:41.910630  7320 leader_election.cc:304] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:41.911540  7674 raft_consensus.cc:2804] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:41.911931  7674 raft_consensus.cc:493] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.912261  7674 raft_consensus.cc:3060] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:41.912352  7320 leader_election.cc:343] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:41.915221  7650 consensus_queue.cc:237] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:41.914266  7551 raft_consensus.cc:359] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.921499  7551 raft_consensus.cc:385] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:41.920179  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 49403.9us
I20260709 13:15:41.921891  7551 raft_consensus.cc:740] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.928325  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:41.928833  7554 tablet_bootstrap.cc:492] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:41.929216  7431 raft_consensus.cc:3060] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.929419  7554 ts_tablet_manager.cc:1403] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.188s	user 0.039s	sys 0.023s
I20260709 13:15:41.934962  7551 consensus_queue.cc:260] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.919027  7503 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:41.946076  7554 raft_consensus.cc:359] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.949374  7554 raft_consensus.cc:385] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.949710  7554 raft_consensus.cc:740] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.956086  7550 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:41.956856  7550 ts_tablet_manager.cc:1403] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.111s	user 0.026s	sys 0.016s
I20260709 13:15:41.958485  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:41.959357  7504 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.960458  7550 raft_consensus.cc:359] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.961196  7550 raft_consensus.cc:385] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:41.961459  7550 raft_consensus.cc:740] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:41.962504  7552 tablet_bootstrap.cc:654] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.962101  7550 consensus_queue.cc:260] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:41.964274  7554 consensus_queue.cc:260] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:41.965783  7550 ts_tablet_manager.cc:1434] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:41.966352  7554 ts_tablet_manager.cc:1434] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 13:15:41.966728  7550 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:41.967291  7554 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:41.919777  7674 raft_consensus.cc:515] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:41.948694  7585 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.976868  7554 tablet_bootstrap.cc:654] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:41.978329  7551 ts_tablet_manager.cc:1434] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.009s
I20260709 13:15:41.979174  7551 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:41.967270  7624 consensus_queue.cc:1048] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:15:41.980197  7674 leader_election.cc:290] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:41.983238  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:41.984076  7428 raft_consensus.cc:3060] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:41.986486  7431 raft_consensus.cc:2468] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:41.988036  7468 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:41.988795  7610 raft_consensus.cc:2804] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:41.989248  7610 raft_consensus.cc:697] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:41.990070  7610 consensus_queue.cc:237] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.001374  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:42.002112  7281 raft_consensus.cc:3060] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.003187  7177 catalog_manager.cc:5697] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.994771  7703 raft_consensus.cc:493] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.982378  7585 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.007406  7703 raft_consensus.cc:515] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.022774  7585 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:42.024076  7703 leader_election.cc:290] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.024556  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:42.025010  7279 raft_consensus.cc:3060] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:42.027386  7248 leader_election.cc:343] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.039296  7504 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:42.045722  7624 consensus_queue.cc:1048] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.049790  7550 tablet_bootstrap.cc:654] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.051188  7428 raft_consensus.cc:2468] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:42.060111  7554 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.060664  7554 ts_tablet_manager.cc:1403] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.094s	user 0.019s	sys 0.012s
I20260709 13:15:42.060616  7320 leader_election.cc:304] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:42.061645  7624 raft_consensus.cc:2804] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.062307  7624 raft_consensus.cc:697] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:42.063592  7554 raft_consensus.cc:359] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.066753  7554 raft_consensus.cc:385] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.063218  7624 consensus_queue.cc:237] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.067088  7554 raft_consensus.cc:740] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.068984  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.069144  7554 consensus_queue.cc:260] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.045204  7281 raft_consensus.cc:2468] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:42.076004  7551 tablet_bootstrap.cc:654] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.087551  7552 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:42.088112  7552 ts_tablet_manager.cc:1403] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.288s	user 0.014s	sys 0.006s
W20260709 13:15:42.088394  7245 leader_election.cc:343] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.088953  7245 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:42.090214  7585 raft_consensus.cc:2749] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.090830  7552 raft_consensus.cc:359] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.092324  7178 catalog_manager.cc:5697] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.095433  7552 raft_consensus.cc:385] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.095842  7552 raft_consensus.cc:740] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.096710  7552 consensus_queue.cc:260] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.099263  7552 ts_tablet_manager.cc:1434] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:15:42.100279  7552 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
W20260709 13:15:42.102790  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 109647us
I20260709 13:15:42.102946  7554 ts_tablet_manager.cc:1434] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.042s	user 0.002s	sys 0.011s
I20260709 13:15:42.104125  7554 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.108105  7394 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:42.109683  7717 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.110198  7717 raft_consensus.cc:697] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:42.111011  7717 consensus_queue.cc:237] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.114378  7624 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.115097  7624 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.116819  7554 tablet_bootstrap.cc:654] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.117730  7624 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:42.114598  7552 tablet_bootstrap.cc:654] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.122973  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.124166  7428 raft_consensus.cc:2393] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:42.119705  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:42.129251  7176 catalog_manager.cc:5697] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.143786  7550 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:42.144309  7550 ts_tablet_manager.cc:1403] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.004s
I20260709 13:15:42.147022  7180 catalog_manager.cc:5697] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.151132  7552 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:42.151711  7552 ts_tablet_manager.cc:1403] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.000s
I20260709 13:15:42.151137  7551 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:42.155390  7551 ts_tablet_manager.cc:1403] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.176s	user 0.011s	sys 0.000s
I20260709 13:15:42.156940  7320 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:42.158363  7674 raft_consensus.cc:3060] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.154306  7552 raft_consensus.cc:359] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.147008  7550 raft_consensus.cc:359] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.160133  7550 raft_consensus.cc:385] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.160408  7550 raft_consensus.cc:740] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.161103  7550 consensus_queue.cc:260] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.163513  7550 ts_tablet_manager.cc:1434] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:15:42.164589  7550 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:42.163618  7179 catalog_manager.cc:5697] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.165150  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.167531  7554 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.168036  7554 ts_tablet_manager.cc:1403] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.064s	user 0.030s	sys 0.019s
W20260709 13:15:42.168788  7468 leader_election.cc:343] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.169356  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.170058  7281 raft_consensus.cc:2468] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:42.166042  7179 catalog_manager.cc:5697] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.170671  7178 catalog_manager.cc:5697] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.172078  7550 tablet_bootstrap.cc:654] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.165779  7674 raft_consensus.cc:2749] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.178577  7468 leader_election.cc:304] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:42.158918  7552 raft_consensus.cc:385] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.181304  7552 raft_consensus.cc:740] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.182085  7552 consensus_queue.cc:260] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.182662  7724 raft_consensus.cc:2804] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.183152  7724 raft_consensus.cc:493] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.158077  7551 raft_consensus.cc:359] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.184267  7551 raft_consensus.cc:385] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.184628  7551 raft_consensus.cc:740] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.185290  7551 consensus_queue.cc:260] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.183596  7724 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.187275  7551 ts_tablet_manager.cc:1434] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.000s
I20260709 13:15:42.188340  7551 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:42.188411  7552 ts_tablet_manager.cc:1434] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:15:42.189497  7552 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:42.170719  7554 raft_consensus.cc:359] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.192045  7554 raft_consensus.cc:385] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.192410  7554 raft_consensus.cc:740] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.194785  7551 tablet_bootstrap.cc:654] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.193336  7554 consensus_queue.cc:260] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.199261  7554 ts_tablet_manager.cc:1434] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:15:42.202575  7554 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.201273  7552 tablet_bootstrap.cc:654] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.206091  7551 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:42.206638  7551 ts_tablet_manager.cc:1403] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.018s	user 0.001s	sys 0.015s
I20260709 13:15:42.211215  7554 tablet_bootstrap.cc:654] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.214529  7552 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:42.215001  7552 ts_tablet_manager.cc:1403] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:15:42.216352  7724 raft_consensus.cc:515] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.220168  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:15:42.222332  7468 leader_election.cc:343] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.225199  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:42.226046  7281 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.237690  7281 raft_consensus.cc:2468] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:42.238799  7724 leader_election.cc:290] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.239594  7468 leader_election.cc:304] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:42.240777  7724 raft_consensus.cc:2804] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:42.241564  7724 raft_consensus.cc:697] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:42.242130  7550 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:42.242300  7554 tablet_bootstrap.cc:492] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.242652  7550 ts_tablet_manager.cc:1403] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.078s	user 0.038s	sys 0.024s
I20260709 13:15:42.242599  7724 consensus_queue.cc:237] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.244223  7552 raft_consensus.cc:359] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.245273  7552 raft_consensus.cc:385] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.245566  7552 raft_consensus.cc:740] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.245591  7550 raft_consensus.cc:359] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.246466  7550 raft_consensus.cc:385] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.246775  7550 raft_consensus.cc:740] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.245455  7551 raft_consensus.cc:359] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.247287  7551 raft_consensus.cc:385] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.247568  7551 raft_consensus.cc:740] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.247478  7550 consensus_queue.cc:260] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.248327  7551 consensus_queue.cc:260] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.249878  7550 ts_tablet_manager.cc:1434] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:42.250522  7551 ts_tablet_manager.cc:1434] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:15:42.250810  7550 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:42.251434  7551 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:42.246559  7552 consensus_queue.cc:260] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.254202  7552 ts_tablet_manager.cc:1434] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.039s	user 0.002s	sys 0.005s
I20260709 13:15:42.242704  7554 ts_tablet_manager.cc:1403] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.019s
I20260709 13:15:42.255575  7552 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:42.256948  7550 tablet_bootstrap.cc:654] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.257272  7551 tablet_bootstrap.cc:654] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.263775  7554 raft_consensus.cc:359] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.273252  7554 raft_consensus.cc:385] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.273656  7554 raft_consensus.cc:740] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.274541  7554 consensus_queue.cc:260] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.272847  7552 tablet_bootstrap.cc:654] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.278083  7554 ts_tablet_manager.cc:1434] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:15:42.280994  7554 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.289901  7554 tablet_bootstrap.cc:654] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.293886  7585 consensus_queue.cc:1048] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.303200  7610 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.303821  7610 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.306339  7610 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:42.308578  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:42.310392  7471 leader_election.cc:343] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:42.314720  7397 consensus_peers.cc:597] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:42.317874  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:42.321781  7468 leader_election.cc:343] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.322398  7468 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:42.320901  7554 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.325855  7554 ts_tablet_manager.cc:1403] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.001s
I20260709 13:15:42.328685  7610 raft_consensus.cc:2749] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.328560  7585 consensus_queue.cc:1048] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.334556  7674 raft_consensus.cc:493] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.335202  7674 raft_consensus.cc:515] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.328529  7554 raft_consensus.cc:359] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.337148  7554 raft_consensus.cc:385] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.337397  7554 raft_consensus.cc:740] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.337594  7674 leader_election.cc:290] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.338023  7554 consensus_queue.cc:260] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.339751  7624 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.339640  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.340745  7281 raft_consensus.cc:2468] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:42.340515  7624 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.342671  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:42.343675  7624 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.344942  7320 leader_election.cc:304] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:42.346899  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.340085  7554 ts_tablet_manager.cc:1434] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:15:42.347884  7504 raft_consensus.cc:2468] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:42.348675  7554 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.352239  7352 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.358953  7624 raft_consensus.cc:2804] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.359404  7624 raft_consensus.cc:493] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.359794  7624 raft_consensus.cc:3060] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.371503  7554 tablet_bootstrap.cc:654] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.380661  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:42.385049  7355 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.389853  7648 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.390419  7648 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.392845  7648 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.393625  7717 consensus_queue.cc:1048] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.403709  7554 tablet_bootstrap.cc:492] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.404191  7554 ts_tablet_manager.cc:1403] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.000s
I20260709 13:15:42.404639  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.406776  7554 raft_consensus.cc:359] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.418218  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.418946  7281 raft_consensus.cc:2468] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:42.420840  7320 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:42.407485  7554 raft_consensus.cc:385] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.427495  7554 raft_consensus.cc:740] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.428375  7554 consensus_queue.cc:260] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.429759  7468 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:42.438288  7554 ts_tablet_manager.cc:1434] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:15:42.439349  7554 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.460788  7703 raft_consensus.cc:2804] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:42.461305  7703 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.461625  7703 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.465291  7554 tablet_bootstrap.cc:654] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:42.471174  7468 leader_election.cc:343] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.471844  7610 consensus_queue.cc:1048] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.473115  7650 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.497936  7650 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.500766  7650 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:42.487972  7727 consensus_queue.cc:1048] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:42.506696  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:42.508816  7471 leader_election.cc:343] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.513084  7428 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.516439  7279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.494906  7703 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.548887  7703 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.575357  7564 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.593621  7717 raft_consensus.cc:493] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:42.612442  7742 log.cc:927] Time spent T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:15:42.648932  7552 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:42.649493  7552 ts_tablet_manager.cc:1403] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.394s	user 0.042s	sys 0.065s
I20260709 13:15:42.652417  7552 raft_consensus.cc:359] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.653095  7552 raft_consensus.cc:385] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.653360  7552 raft_consensus.cc:740] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.653980  7552 consensus_queue.cc:260] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.663573  7552 ts_tablet_manager.cc:1434] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:15:42.644541  7564 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.664582  7552 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:42.671471  7356 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.644737  7717 raft_consensus.cc:515] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.684976  7564 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:42.693436  7552 tablet_bootstrap.cc:654] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.643807  7624 raft_consensus.cc:515] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.715984  7717 leader_election.cc:290] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.718670  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.720109  7281 raft_consensus.cc:2393] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:42.735754  7747 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.736697  7747 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.739364  7747 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:15:42.774904  7751 log.cc:927] Time spent T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:42.788578  7717 consensus_queue.cc:1048] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:42.710350  7468 leader_election.cc:343] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:42.807650  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.814198  7624 leader_election.cc:290] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:42.816957  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:42.818190  7503 raft_consensus.cc:2468] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:42.822567  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:42.833946  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:42.839986  7357 raft_consensus.cc:3060] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.858515  7468 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:42.860112  7724 raft_consensus.cc:2749] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:42.861764  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 153636us
W20260709 13:15:42.877871  7550 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.179s	sys 0.297s
W20260709 13:15:42.878278  7550 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.179s	sys 0.297s
W20260709 13:15:42.877892  7551 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.171s	sys 0.314s
W20260709 13:15:42.879026  7551 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.175s	sys 0.314s
I20260709 13:15:42.903618  7551 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:42.904204  7551 ts_tablet_manager.cc:1403] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.653s	user 0.182s	sys 0.314s
W20260709 13:15:42.908342  7749 log.cc:927] Time spent T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:15:42.912217  7551 raft_consensus.cc:359] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.914346  7551 raft_consensus.cc:385] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.914791  7564 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.915311  7564 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:42.915778  7717 consensus_queue.cc:1048] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:42.917889  7564 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:42.920101  7551 raft_consensus.cc:740] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.921440  7554 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:42.921938  7554 ts_tablet_manager.cc:1403] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.483s	user 0.055s	sys 0.064s
I20260709 13:15:42.925746  7551 consensus_queue.cc:260] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.926380  7550 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:42.926868  7550 ts_tablet_manager.cc:1403] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.676s	user 0.185s	sys 0.298s
I20260709 13:15:42.928807  7554 raft_consensus.cc:359] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.929564  7554 raft_consensus.cc:385] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.929898  7554 raft_consensus.cc:740] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.931001  7550 raft_consensus.cc:359] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.930919  7554 consensus_queue.cc:260] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:42.936026  7550 raft_consensus.cc:385] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.928769  7551 ts_tablet_manager.cc:1434] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 13:15:42.936435  7550 raft_consensus.cc:740] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.937378  7551 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:42.937211  7550 consensus_queue.cc:260] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:42.946295  7552 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:42.946961  7552 ts_tablet_manager.cc:1403] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.283s	user 0.048s	sys 0.053s
I20260709 13:15:42.947372  7554 ts_tablet_manager.cc:1434] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.006s
I20260709 13:15:42.949731  7550 ts_tablet_manager.cc:1434] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 13:15:42.952843  7550 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:42.955991  7674 raft_consensus.cc:2749] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:42.964757  7554 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:42.966379  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:42.968806  7552 raft_consensus.cc:359] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.971652  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:42.972342  7281 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:42.975111  7320 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:42.978014  7624 raft_consensus.cc:2804] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:42.978400  7624 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:15:42.978849  7624 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:42.961457  7179 catalog_manager.cc:5697] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.987726  7757 consensus_queue.cc:1048] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:15:42.989190  7552 raft_consensus.cc:385] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:42.989696  7552 raft_consensus.cc:740] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:42.990527  7552 consensus_queue.cc:260] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:42.992736  7551 tablet_bootstrap.cc:654] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:42.999750  7281 raft_consensus.cc:2468] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:43.006656  7552 ts_tablet_manager.cc:1434] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.003s
I20260709 13:15:43.006513  7717 consensus_queue.cc:1048] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.008255  7552 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:42.994256  7624 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.009217  7352 raft_consensus.cc:1240] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:43.007931  7554 tablet_bootstrap.cc:654] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.015472  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:43.016005  7281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:43.016201  7503 raft_consensus.cc:3060] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.016495  7717 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.017194  7717 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.018298  7552 tablet_bootstrap.cc:654] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.020826  7717 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.020709  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.020715  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.021917  7282 raft_consensus.cc:2468] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:43.022562  7356 raft_consensus.cc:2468] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:43.023993  7394 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:43.016616  7281 raft_consensus.cc:3060] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.024958  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:43.027793  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.028405  7504 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:43.032197  7691 raft_consensus.cc:2804] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.032606  7691 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.032955  7691 raft_consensus.cc:3060] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.045639  7691 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.049458  7629 consensus_queue.cc:1059] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.045825  7703 consensus_queue.cc:1048] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.050210  7691 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.052006  7320 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.053629  7757 raft_consensus.cc:2749] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.053973  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:43.054383  7550 tablet_bootstrap.cc:654] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.054571  7278 raft_consensus.cc:3060] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.056496  7624 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:43.058734  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:43.061972  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:43.062597  7504 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.079849  7320 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.085741  7757 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:43.088617  7757 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:15:43.089157  7757 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.093425  7503 raft_consensus.cc:2468] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:43.095247  7320 leader_election.cc:304] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:43.096212  7624 raft_consensus.cc:2804] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.096784  7624 raft_consensus.cc:697] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:43.098044  7281 raft_consensus.cc:2468] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:43.100675  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:43.101553  7357 raft_consensus.cc:3060] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.097846  7624 consensus_queue.cc:237] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.109565  7504 raft_consensus.cc:2468] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:43.116834  7320 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.114861  7757 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.117846  7552 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:43.118113  7674 raft_consensus.cc:2804] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:43.118487  7552 ts_tablet_manager.cc:1403] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.111s	user 0.033s	sys 0.067s
I20260709 13:15:43.122988  7551 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:43.123452  7551 ts_tablet_manager.cc:1403] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.004s
I20260709 13:15:43.121780  7674 raft_consensus.cc:697] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:43.124152  7278 raft_consensus.cc:2468] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:43.124866  7674 consensus_queue.cc:237] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.125407  7394 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:43.130682  7357 raft_consensus.cc:2468] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:43.131316  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:15:43.141223  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:43.146333  7571 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.146920  7571 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.147644  7552 raft_consensus.cc:359] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.148351  7552 raft_consensus.cc:385] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.148615  7552 raft_consensus.cc:740] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.148739  7717 raft_consensus.cc:2804] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.151949  7717 raft_consensus.cc:697] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:43.152729  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:43.149210  7554 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:43.149305  7552 consensus_queue.cc:260] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.153416  7504 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.153885  7554 ts_tablet_manager.cc:1403] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.189s	user 0.016s	sys 0.009s
I20260709 13:15:43.150662  7571 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.155354  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.158478  7554 raft_consensus.cc:359] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.159456  7554 raft_consensus.cc:385] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.159873  7554 raft_consensus.cc:740] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.155965  7757 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:43.164202  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.157104  7503 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:43.177240  7245 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:43.178156  7585 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.178521  7585 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.178853  7585 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.169971  7554 consensus_queue.cc:260] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.197431  7554 ts_tablet_manager.cc:1434] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:15:43.201673  7554 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
W20260709 13:15:43.196610  7468 leader_election.cc:343] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.196375  7717 consensus_queue.cc:237] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.204587  7551 raft_consensus.cc:359] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.209822  7551 raft_consensus.cc:385] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.198686  7357 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:43.226681  7504 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:43.210250  7551 raft_consensus.cc:740] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.234711  7551 consensus_queue.cc:260] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.235549  7552 ts_tablet_manager.cc:1434] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.117s	user 0.022s	sys 0.021s
I20260709 13:15:43.247280  7552 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:43.248977  7551 ts_tablet_manager.cc:1434] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.000s
I20260709 13:15:43.234421  7691 raft_consensus.cc:3055] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:43.252099  7691 raft_consensus.cc:740] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:43.252998  7691 consensus_queue.cc:260] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.254245  7691 raft_consensus.cc:3060] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.255535  7554 tablet_bootstrap.cc:654] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:43.234687  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 47147us
I20260709 13:15:43.256392  7320 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.257238  7550 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:43.257815  7550 ts_tablet_manager.cc:1403] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.307s	user 0.020s	sys 0.040s
I20260709 13:15:43.260437  7624 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:43.261016  7624 raft_consensus.cc:697] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:43.261880  7624 consensus_queue.cc:237] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.263835  7571 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.264430  7571 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.267127  7571 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.267876  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.268584  7504 raft_consensus.cc:2468] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:43.268606  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.270088  7245 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
W20260709 13:15:43.239864  7471 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 54313.6us
I20260709 13:15:43.277984  7551 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:43.273078  7770 raft_consensus.cc:2804] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.258857  7585 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.293738  7585 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.295079  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
W20260709 13:15:43.295840  7248 leader_election.cc:343] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.296064  7504 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.289315  7770 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.299938  7770 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.302625  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:43.303530  7356 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.280771  7773 consensus_queue.cc:1048] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:15:43.314895  7550 raft_consensus.cc:359] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.335318  7550 raft_consensus.cc:385] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.338829  7550 raft_consensus.cc:740] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.339645  7550 consensus_queue.cc:260] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.341547  7770 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.331588  7551 tablet_bootstrap.cc:654] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.342082  7550 ts_tablet_manager.cc:1434] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.000s
I20260709 13:15:43.347280  7550 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:43.368368  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:43.368997  7503 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.373787  7176 catalog_manager.cc:5697] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.380779  7501 raft_consensus.cc:1240] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:43.374701  7504 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:43.375417  7505 raft_consensus.cc:1240] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:43.374364  7356 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:43.385645  7248 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:43.386516  7622 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.386997  7622 raft_consensus.cc:697] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:43.387164  7468 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.387840  7622 consensus_queue.cc:237] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.382826  7550 tablet_bootstrap.cc:654] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.389995  7552 tablet_bootstrap.cc:654] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.390640  7773 consensus_queue.cc:1048] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.391693  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.379948  7431 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.375667  7503 raft_consensus.cc:2468] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:43.401825  7703 consensus_queue.cc:1048] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:15:43.419189  7554 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:43.419770  7554 ts_tablet_manager.cc:1403] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.218s	user 0.034s	sys 0.032s
I20260709 13:15:43.422449  7554 raft_consensus.cc:359] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.423110  7554 raft_consensus.cc:385] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.423360  7554 raft_consensus.cc:740] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.424400  7554 consensus_queue.cc:260] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.433163  7551 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:43.433949  7551 ts_tablet_manager.cc:1403] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.156s	user 0.020s	sys 0.003s
I20260709 13:15:43.440579  7554 ts_tablet_manager.cc:1434] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:15:43.441937  7554 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:43.445731  7778 raft_consensus.cc:2804] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.446622  7778 raft_consensus.cc:697] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:43.447741  7778 consensus_queue.cc:237] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:43.456341  7396 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.457175  7770 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.461500  7245 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:43.462487  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:43.463451  7585 raft_consensus.cc:2804] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.463969  7585 raft_consensus.cc:697] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:43.464813  7585 consensus_queue.cc:237] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:43.465994  7248 leader_election.cc:343] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:43.468977  7471 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 59317.9us
W20260709 13:15:43.474057  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 56950.8us
I20260709 13:15:43.497360  7550 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:43.497942  7550 ts_tablet_manager.cc:1403] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.010s
I20260709 13:15:43.500509  7552 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:43.501055  7552 ts_tablet_manager.cc:1403] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.258s	user 0.020s	sys 0.002s
I20260709 13:15:43.502807  7691 consensus_queue.cc:1059] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.504268  7691 raft_consensus.cc:3055] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:43.504352  7757 raft_consensus.cc:493] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.504665  7691 raft_consensus.cc:740] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:43.504976  7757 raft_consensus.cc:515] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.505620  7691 consensus_queue.cc:260] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.507087  7757 leader_election.cc:290] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:43.508109  7703 consensus_queue.cc:1048] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.510617  7550 raft_consensus.cc:359] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.511338  7550 raft_consensus.cc:385] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.511641  7550 raft_consensus.cc:740] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.512125  7551 raft_consensus.cc:359] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.512251  7550 consensus_queue.cc:260] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.511135  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:43.513875  7550 ts_tablet_manager.cc:1434] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:15:43.513545  7552 raft_consensus.cc:359] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.508879  7691 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:43.514551  7320 leader_election.cc:343] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.514832  7550 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:43.515805  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.516458  7282 raft_consensus.cc:2468] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:43.517750  7552 raft_consensus.cc:385] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.518224  7552 raft_consensus.cc:740] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.517787  7320 leader_election.cc:304] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.521077  7554 tablet_bootstrap.cc:654] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.521513  7394 leader_election.cc:304] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:43.522572  7629 raft_consensus.cc:2749] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.519212  7552 consensus_queue.cc:260] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.512871  7551 raft_consensus.cc:385] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.523914  7551 raft_consensus.cc:740] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.524839  7757 raft_consensus.cc:2804] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.525305  7757 raft_consensus.cc:493] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.525826  7757 raft_consensus.cc:3060] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.525761  7551 consensus_queue.cc:260] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.530344  7552 ts_tablet_manager.cc:1434] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.001s
I20260709 13:15:43.531327  7551 ts_tablet_manager.cc:1434] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.002s
I20260709 13:15:43.531374  7552 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:43.536451  7703 raft_consensus.cc:493] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.537124  7703 raft_consensus.cc:515] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.539245  7703 leader_election.cc:290] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:43.541939  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.542763  7282 raft_consensus.cc:2468] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:43.545965  7430 raft_consensus.cc:1275] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.546168  7468 leader_election.cc:304] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:43.547652  7551 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:43.547681  7674 consensus_queue.cc:1048] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:43.551826  7703 raft_consensus.cc:2804] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.552332  7703 raft_consensus.cc:493] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.552719  7703 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.557742  7778 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.558317  7778 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.559926  7503 raft_consensus.cc:1275] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.561565  7778 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.562988  7747 consensus_queue.cc:1048] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.563158  7279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.560487  7178 catalog_manager.cc:5697] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:43.565661  7468 leader_election.cc:343] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.565047  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:43.568392  7471 leader_election.cc:343] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.568846  7471 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:43.569942  7789 raft_consensus.cc:2749] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.569999  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:43.576157  7179 catalog_manager.cc:5697] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.581081  7703 raft_consensus.cc:515] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.578668  7179 catalog_manager.cc:5697] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 2, leader changed from 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.593582  7550 tablet_bootstrap.cc:654] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.598765  7551 tablet_bootstrap.cc:654] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.594686  7179 catalog_manager.cc:5697] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 2, leader changed from 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.577315  7552 tablet_bootstrap.cc:654] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:43.595721  7468 leader_election.cc:343] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.603385  7757 raft_consensus.cc:515] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.603385  7703 leader_election.cc:290] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:43.607832  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:43.608245  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:43.608466  7282 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.610255  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:43.611307  7278 raft_consensus.cc:3060] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.621769  7757 leader_election.cc:290] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:43.623941  7554 tablet_bootstrap.cc:492] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:43.623858  7282 raft_consensus.cc:2468] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:43.624531  7554 ts_tablet_manager.cc:1403] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.183s	user 0.013s	sys 0.001s
I20260709 13:15:43.626015  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:15:43.627661  7320 leader_election.cc:343] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.627267  7554 raft_consensus.cc:359] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.629338  7554 raft_consensus.cc:385] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:43.629474  7468 leader_election.cc:343] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.629654  7554 raft_consensus.cc:740] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.630941  7554 consensus_queue.cc:260] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.632398  7468 leader_election.cc:304] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.633334  7787 raft_consensus.cc:2804] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.633805  7787 raft_consensus.cc:697] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:43.634601  7787 consensus_queue.cc:237] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.635553  7178 catalog_manager.cc:5697] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.637524  7178 catalog_manager.cc:5697] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.651713  7691 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.652380  7691 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.652992  7552 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:43.653483  7552 ts_tablet_manager.cc:1403] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.000s
I20260709 13:15:43.654567  7691 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:43.656404  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.657845  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.658708  7282 raft_consensus.cc:2393] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:43.660162  7551 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:43.660688  7551 ts_tablet_manager.cc:1403] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.113s	user 0.006s	sys 0.013s
I20260709 13:15:43.661577  7554 ts_tablet_manager.cc:1434] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:15:43.662549  7554 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:43.663552  7551 raft_consensus.cc:359] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.664335  7551 raft_consensus.cc:385] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.664600  7551 raft_consensus.cc:740] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.665462  7551 consensus_queue.cc:260] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.676777  7428 raft_consensus.cc:1275] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.700589  7179 catalog_manager.cc:5697] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.702481  7747 consensus_queue.cc:1048] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:43.702805  7551 ts_tablet_manager.cc:1434] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.004s
I20260709 13:15:43.674475  7717 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.704178  7551 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:43.704015  7717 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.710036  7717 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:43.672096  7554 tablet_bootstrap.cc:654] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.710762  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.711524  7356 raft_consensus.cc:2468] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:43.671115  7674 consensus_queue.cc:1048] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.712857  7397 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:43.715327  7551 tablet_bootstrap.cc:654] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.666742  7394 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:43.716574  7691 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:43.716655  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 49987.8us
I20260709 13:15:43.669791  7278 raft_consensus.cc:2468] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:43.718276  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.718480  7629 raft_consensus.cc:493] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.718950  7503 raft_consensus.cc:2468] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:43.719017  7629 raft_consensus.cc:515] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.719835  7757 consensus_queue.cc:1048] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.721405  7629 leader_election.cc:290] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:43.723806  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.724798  7356 raft_consensus.cc:2393] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:43.687311  7550 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:43.726704  7550 ts_tablet_manager.cc:1403] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.212s	user 0.016s	sys 0.013s
I20260709 13:15:43.728679  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:43.729601  7278 raft_consensus.cc:2393] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:43.731195  7394 leader_election.cc:304] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:43.732609  7550 raft_consensus.cc:359] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.668113  7552 raft_consensus.cc:359] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.733563  7629 raft_consensus.cc:3060] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.733755  7552 raft_consensus.cc:385] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.734297  7552 raft_consensus.cc:740] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.732125  7320 leader_election.cc:304] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:43.736511  7674 raft_consensus.cc:2804] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.736994  7674 raft_consensus.cc:697] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:43.737941  7674 consensus_queue.cc:237] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.739637  7771 consensus_queue.cc:1048] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.715750  7717 raft_consensus.cc:2804] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:43.740677  7717 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.741065  7717 raft_consensus.cc:3060] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.746512  7502 raft_consensus.cc:1275] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.737824  7552 consensus_queue.cc:260] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.729516  7691 raft_consensus.cc:2749] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.733604  7550 raft_consensus.cc:385] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.753850  7550 raft_consensus.cc:740] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.751305  7629 raft_consensus.cc:2749] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.755883  7550 consensus_queue.cc:260] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:43.757262  7552 ts_tablet_manager.cc:1434] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.104s	user 0.014s	sys 0.003s
I20260709 13:15:43.758368  7550 ts_tablet_manager.cc:1434] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:15:43.759202  7550 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:43.760654  7747 consensus_queue.cc:1048] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.770278  7552 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:43.795116  7550 tablet_bootstrap.cc:654] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.806205  7554 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:43.806740  7554 ts_tablet_manager.cc:1403] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.144s	user 0.012s	sys 0.002s
I20260709 13:15:43.811167  7792 consensus_queue.cc:1048] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:43.811859  7806 log.cc:927] Time spent T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:15:43.814841  7717 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.816224  7554 raft_consensus.cc:359] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.817034  7554 raft_consensus.cc:385] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.817358  7554 raft_consensus.cc:740] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.817776  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:43.818397  7356 raft_consensus.cc:3060] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.818105  7554 consensus_queue.cc:260] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:43.820997  7554 ts_tablet_manager.cc:1434] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:15:43.822005  7554 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:43.825047  7771 consensus_queue.cc:1048] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.843684  7552 tablet_bootstrap.cc:654] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.866885  7789 consensus_queue.cc:1048] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:43.869951  7717 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:43.876101  7789 raft_consensus.cc:493] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.877761  7789 raft_consensus.cc:515] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.880426  7622 consensus_queue.cc:1048] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:43.878733  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:43.881632  7503 raft_consensus.cc:3060] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.878979  7551 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:43.882375  7551 ts_tablet_manager.cc:1403] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.178s	user 0.015s	sys 0.009s
I20260709 13:15:43.882984  7356 raft_consensus.cc:2468] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:43.882915  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:43.890805  7554 tablet_bootstrap.cc:654] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.895690  7551 raft_consensus.cc:359] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.897459  7397 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:43.900064  7551 raft_consensus.cc:385] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.902781  7551 raft_consensus.cc:740] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
W20260709 13:15:43.898132  7471 leader_election.cc:343] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.904760  7551 consensus_queue.cc:260] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.905668  7771 raft_consensus.cc:2804] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.908417  7771 raft_consensus.cc:697] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:43.909314  7771 consensus_queue.cc:237] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.919795  7552 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:43.920492  7552 ts_tablet_manager.cc:1403] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.010s
I20260709 13:15:43.923910  7717 raft_consensus.cc:493] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.924484  7717 raft_consensus.cc:515] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:43.924714  7552 raft_consensus.cc:359] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.925484  7552 raft_consensus.cc:385] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.925824  7552 raft_consensus.cc:740] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.926748  7717 leader_election.cc:290] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:43.928390  7552 consensus_queue.cc:260] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:43.928669  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:43.930569  7552 ts_tablet_manager.cc:1434] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
W20260709 13:15:43.930817  7394 leader_election.cc:343] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.931730  7552 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:43.932330  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:43.934944  7397 leader_election.cc:343] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:43.935406  7397 leader_election.cc:304] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:43.938297  7717 raft_consensus.cc:2749] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:43.939572  7552 tablet_bootstrap.cc:654] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.939075  7177 catalog_manager.cc:5697] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.948242  7551 ts_tablet_manager.cc:1434] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:15:43.951254  7551 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:43.980731  7789 leader_election.cc:290] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:43.998844  7550 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:43.999719  7550 ts_tablet_manager.cc:1403] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.241s	user 0.012s	sys 0.009s
I20260709 13:15:44.007860  7550 raft_consensus.cc:359] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.009281  7550 raft_consensus.cc:385] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.009490  7550 raft_consensus.cc:740] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.009930  7550 consensus_queue.cc:260] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.013530  7552 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:44.014073  7552 ts_tablet_manager.cc:1403] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.083s	user 0.024s	sys 0.008s
I20260709 13:15:44.015211  7503 raft_consensus.cc:2468] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:44.016829  7552 raft_consensus.cc:359] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.020200  7552 raft_consensus.cc:385] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.020517  7552 raft_consensus.cc:740] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.021608  7550 ts_tablet_manager.cc:1434] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.002s
I20260709 13:15:44.022085  7552 consensus_queue.cc:260] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.022496  7550 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:44.028875  7552 ts_tablet_manager.cc:1434] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:15:44.030141  7757 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.030666  7757 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.032663  7757 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.033978  7552 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:44.036164  7550 tablet_bootstrap.cc:654] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.037503  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.038311  7505 raft_consensus.cc:2393] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:44.040117  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.043215  7552 tablet_bootstrap.cc:654] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.047027  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:44.054150  7554 tablet_bootstrap.cc:492] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:44.047484  7352 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.056205  7554 ts_tablet_manager.cc:1403] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.234s	user 0.018s	sys 0.019s
I20260709 13:15:44.059530  7622 consensus_queue.cc:1048] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:44.043586  7468 leader_election.cc:343] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.060700  7468 leader_election.cc:304] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:44.061499  7789 raft_consensus.cc:2749] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.052271  7551 tablet_bootstrap.cc:654] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.059183  7554 raft_consensus.cc:359] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.063119  7554 raft_consensus.cc:385] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.067366  7554 raft_consensus.cc:740] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.070087  7554 consensus_queue.cc:260] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.073477  7789 raft_consensus.cc:493] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.074129  7789 raft_consensus.cc:515] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.075124  7320 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:44.076810  7789 leader_election.cc:290] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:44.077595  7585 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.077603  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:44.078222  7585 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.078697  7356 raft_consensus.cc:2468] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:44.080394  7471 leader_election.cc:304] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:44.080947  7585 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:44.081445  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.082203  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:44.082370  7503 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:44.084204  7245 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
W20260709 13:15:44.085129  7248 leader_election.cc:343] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.085666  7778 raft_consensus.cc:2804] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.086229  7778 raft_consensus.cc:493] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.086601  7778 raft_consensus.cc:3060] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.088624  7622 raft_consensus.cc:2804] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.089042  7622 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.089438  7622 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.089287  7787 consensus_queue.cc:1048] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.086699  7554 ts_tablet_manager.cc:1434] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:15:44.092247  7554 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:44.092483  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.093129  7428 raft_consensus.cc:2468] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:44.107229  7552 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:44.108078  7552 ts_tablet_manager.cc:1403] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.021s
I20260709 13:15:44.111949  7552 raft_consensus.cc:359] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.112921  7552 raft_consensus.cc:385] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.114003  7552 raft_consensus.cc:740] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.114694  7552 consensus_queue.cc:260] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.132719  7552 ts_tablet_manager.cc:1434] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 13:15:44.133929  7552 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:44.137259  7789 raft_consensus.cc:493] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.138024  7554 tablet_bootstrap.cc:654] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.137842  7789 raft_consensus.cc:515] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.146274  7789 leader_election.cc:290] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.147361  7551 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:44.147915  7551 ts_tablet_manager.cc:1403] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.199s	user 0.008s	sys 0.033s
I20260709 13:15:44.148237  7552 tablet_bootstrap.cc:654] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.148487  7550 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:44.148347  7778 raft_consensus.cc:515] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.151388  7778 leader_election.cc:290] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:44.149011  7550 ts_tablet_manager.cc:1403] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.127s	user 0.008s	sys 0.031s
I20260709 13:15:44.151049  7551 raft_consensus.cc:359] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.152760  7551 raft_consensus.cc:385] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.153087  7551 raft_consensus.cc:740] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.153409  7622 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.153764  7551 consensus_queue.cc:260] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.155820  7622 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:44.157774  7551 ts_tablet_manager.cc:1434] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:44.159444  7551 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:44.158872  7550 raft_consensus.cc:359] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.160012  7550 raft_consensus.cc:385] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.160338  7550 raft_consensus.cc:740] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.161101  7550 consensus_queue.cc:260] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.215725  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:44.218190  7278 raft_consensus.cc:2468] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:44.223037  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:44.224164  7356 raft_consensus.cc:3060] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.226272  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:44.229419  7468 leader_election.cc:304] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:44.231127  7550 ts_tablet_manager.cc:1434] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.000s
I20260709 13:15:44.241637  7550 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:44.242041  7703 raft_consensus.cc:2804] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.242604  7703 raft_consensus.cc:493] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.243042  7703 raft_consensus.cc:3060] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.242728  7787 consensus_queue.cc:1048] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.233577  7552 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:44.234637  7622 consensus_queue.cc:1048] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.245990  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90ec340d335041b0bc58e0bcdf0b8970" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:44.236302  7747 raft_consensus.cc:2749] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.256227  7552 ts_tablet_manager.cc:1403] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.008s
I20260709 13:15:44.253969  7179 catalog_manager.cc:5697] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.247085  7430 raft_consensus.cc:3060] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.236346  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:44.282347  7248 leader_election.cc:343] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.290215  7551 tablet_bootstrap.cc:654] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.301913  7552 raft_consensus.cc:359] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.307777  7552 raft_consensus.cc:385] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.303249  7550 tablet_bootstrap.cc:654] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.308182  7552 raft_consensus.cc:740] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.322343  7552 consensus_queue.cc:260] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.326719  7552 ts_tablet_manager.cc:1434] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.004s
I20260709 13:15:44.328755  7552 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:44.332170  7747 consensus_queue.cc:1048] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:44.333168  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 217145us
I20260709 13:15:44.335569  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:44.336303  7505 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.341665  7771 consensus_queue.cc:1048] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.344048  7674 consensus_queue.cc:1048] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.348717  7429 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.379163  7552 tablet_bootstrap.cc:654] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.387847  7554 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:44.388648  7554 ts_tablet_manager.cc:1403] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.297s	user 0.011s	sys 0.003s
I20260709 13:15:44.405405  7356 raft_consensus.cc:2468] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:44.415261  7554 raft_consensus.cc:359] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.417510  7554 raft_consensus.cc:385] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.419724  7554 raft_consensus.cc:740] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.420428  7505 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:44.424968  7703 raft_consensus.cc:515] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.421689  7554 consensus_queue.cc:260] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.456485  7554 ts_tablet_manager.cc:1434] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.003s
I20260709 13:15:44.450760  7787 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.470445  7792 raft_consensus.cc:493] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.471216  7792 raft_consensus.cc:515] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.471693  7703 leader_election.cc:290] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.476382  7551 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:44.477290  7551 ts_tablet_manager.cc:1403] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.319s	user 0.011s	sys 0.009s
I20260709 13:15:44.483870  7554 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:44.490814  7622 raft_consensus.cc:493] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.492950  7471 leader_election.cc:304] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:44.495718  7610 raft_consensus.cc:2804] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.496426  7610 raft_consensus.cc:697] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:44.497622  7610 consensus_queue.cc:237] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.499224  7554 tablet_bootstrap.cc:654] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.481097  7787 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:44.504207  7248 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 53227.4us
W20260709 13:15:44.481483  7468 leader_election.cc:343] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.500882  7622 raft_consensus.cc:515] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.519130  7430 raft_consensus.cc:2468] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
W20260709 13:15:44.531191  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 77955.1us
I20260709 13:15:44.543992  7622 leader_election.cc:290] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
W20260709 13:15:44.545244  7831 log.cc:927] Time spent T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:15:44.531924  7792 leader_election.cc:290] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:44.533196  7717 raft_consensus.cc:493] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.534722  7787 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:44.537631  7550 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:44.559036  7550 ts_tablet_manager.cc:1403] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.318s	user 0.007s	sys 0.020s
I20260709 13:15:44.539368  7551 raft_consensus.cc:359] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.565934  7551 raft_consensus.cc:385] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.566330  7551 raft_consensus.cc:740] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.572942  7552 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:44.573534  7552 ts_tablet_manager.cc:1403] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.245s	user 0.011s	sys 0.010s
W20260709 13:15:44.577252  7830 log.cc:927] Time spent T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
I20260709 13:15:44.565073  7717 raft_consensus.cc:515] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.582243  7717 leader_election.cc:290] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.561734  7747 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.583078  7747 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.582027  7551 consensus_queue.cc:260] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.585238  7551 ts_tablet_manager.cc:1434] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.108s	user 0.001s	sys 0.008s
I20260709 13:15:44.585695  7747 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.586709  7551 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:44.588598  7554 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:44.589107  7554 ts_tablet_manager.cc:1403] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.012s
I20260709 13:15:44.587284  7552 raft_consensus.cc:359] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.589448  7552 raft_consensus.cc:385] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.589771  7552 raft_consensus.cc:740] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.590330  7778 consensus_queue.cc:1048] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:44.590636  7552 consensus_queue.cc:260] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.592955  7552 ts_tablet_manager.cc:1434] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:15:44.593828  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.594714  7503 raft_consensus.cc:2393] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:44.594848  7552 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:44.596746  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:44.597685  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:44.598313  7430 raft_consensus.cc:3060] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.599506  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:44.603374  7771 consensus_queue.cc:1048] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:44.603955  7554 raft_consensus.cc:359] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.605085  7554 raft_consensus.cc:385] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.605423  7554 raft_consensus.cc:740] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.613636  7554 consensus_queue.cc:260] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.575428  7550 raft_consensus.cc:359] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.620890  7550 raft_consensus.cc:385] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.621347  7550 raft_consensus.cc:740] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.633715  7550 consensus_queue.cc:260] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:44.631131  7248 leader_election.cc:343] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.666821  7554 ts_tablet_manager.cc:1434] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 13:15:44.670332  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:44.670420  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:44.673085  7550 ts_tablet_manager.cc:1434] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.001s
W20260709 13:15:44.673938  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:44.690327  7552 tablet_bootstrap.cc:654] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.696060  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.702924  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:44.704977  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.705500  7792 raft_consensus.cc:493] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.705781  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.706073  7792 raft_consensus.cc:515] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.708353  7792 leader_election.cc:290] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:44.709489  7245 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:44.710064  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:44.713757  7554 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
W20260709 13:15:44.715579  7471 leader_election.cc:343] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.718830  7280 raft_consensus.cc:2393] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:44.720319  7844 raft_consensus.cc:2804] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.720906  7844 raft_consensus.cc:697] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:44.722162  7844 consensus_queue.cc:237] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.723786  7610 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.724375  7610 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.725723  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:44.703394  7430 raft_consensus.cc:2468] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
W20260709 13:15:44.697216  7248 leader_election.cc:343] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.736435  7550 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:44.742790  7554 tablet_bootstrap.cc:654] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.745534  7551 tablet_bootstrap.cc:654] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.729332  7276 raft_consensus.cc:3060] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.760512  7552 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:44.772518  7552 ts_tablet_manager.cc:1403] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.179s	user 0.013s	sys 0.009s
I20260709 13:15:44.748008  7610 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:44.756245  7674 raft_consensus.cc:493] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.786183  7178 catalog_manager.cc:5697] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:44.787928  7674 raft_consensus.cc:515] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:15:44.754601  7248 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 61261.3us
I20260709 13:15:44.798149  7320 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:44.798851  7550 tablet_bootstrap.cc:654] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.800817  7757 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:44.755807  7245 leader_election.cc:343] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:44.818007  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 55995.1us
I20260709 13:15:44.834268  7552 raft_consensus.cc:359] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.839329  7552 raft_consensus.cc:385] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.839987  7552 raft_consensus.cc:740] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.840866  7552 consensus_queue.cc:260] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.839166  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.842927  7245 leader_election.cc:304] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:44.837441  7674 leader_election.cc:290] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:44.843683  7552 ts_tablet_manager.cc:1434] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:15:44.845804  7554 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:44.846367  7554 ts_tablet_manager.cc:1403] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.133s	user 0.019s	sys 0.011s
W20260709 13:15:44.847229  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 134852us
I20260709 13:15:44.827874  7468 leader_election.cc:304] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:44.848974  7844 raft_consensus.cc:2749] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.861884  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.862730  7503 raft_consensus.cc:2468] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
W20260709 13:15:44.864542  7320 leader_election.cc:343] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.864318  7554 raft_consensus.cc:359] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.865528  7554 raft_consensus.cc:385] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.865828  7554 raft_consensus.cc:740] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.867061  7551 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:44.867658  7551 ts_tablet_manager.cc:1403] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.281s	user 0.020s	sys 0.000s
I20260709 13:15:44.867462  7554 consensus_queue.cc:260] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:44.869683  7552 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:44.872269  7276 raft_consensus.cc:2468] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:44.872198  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.874948  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:44.875182  7394 leader_election.cc:304] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:44.876700  7771 raft_consensus.cc:2749] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:44.881793  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:44.882373  7550 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:44.882898  7550 ts_tablet_manager.cc:1403] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.185s	user 0.008s	sys 0.012s
I20260709 13:15:44.883329  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
W20260709 13:15:44.883978  7245 leader_election.cc:343] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.884096  7320 leader_election.cc:304] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:44.884547  7245 leader_election.cc:304] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:44.885492  7842 raft_consensus.cc:2749] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.889612  7857 raft_consensus.cc:2804] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.890202  7857 raft_consensus.cc:697] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:44.891049  7857 consensus_queue.cc:237] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:44.898008  7674 raft_consensus.cc:2804] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.898545  7674 raft_consensus.cc:493] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.898906  7674 raft_consensus.cc:3060] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.900848  7554 ts_tablet_manager.cc:1434] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 13:15:44.922689  7554 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:44.930845  7551 raft_consensus.cc:359] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.931521  7551 raft_consensus.cc:385] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.917145  7674 raft_consensus.cc:515] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.936491  7674 leader_election.cc:290] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:44.937494  7550 raft_consensus.cc:359] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.938230  7550 raft_consensus.cc:385] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.938529  7550 raft_consensus.cc:740] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
W20260709 13:15:44.932130  7245 leader_election.cc:343] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.941417  7792 raft_consensus.cc:493] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.941880  7792 raft_consensus.cc:515] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.945237  7552 tablet_bootstrap.cc:654] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.947009  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:44.948019  7245 leader_election.cc:343] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.949826  7245 leader_election.cc:304] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:44.951189  7839 raft_consensus.cc:2749] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.950853  7789 consensus_queue.cc:1048] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:44.952229  7248 leader_election.cc:343] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.954031  7792 leader_election.cc:290] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:44.956183  7554 tablet_bootstrap.cc:654] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.961401  7757 raft_consensus.cc:2749] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.948928  7550 consensus_queue.cc:260] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:44.931836  7551 raft_consensus.cc:740] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
W20260709 13:15:44.916219  7468 leader_election.cc:343] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.965350  7551 consensus_queue.cc:260] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:44.967201  7468 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:44.972155  7179 catalog_manager.cc:5697] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:44.978983  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 62807.5us
I20260709 13:15:44.981611  7550 ts_tablet_manager.cc:1434] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.002s
I20260709 13:15:44.982841  7550 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:44.991714  7857 raft_consensus.cc:2749] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.983021  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:44.998211  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.000196  7551 ts_tablet_manager.cc:1434] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.132s	user 0.008s	sys 0.000s
I20260709 13:15:45.001183  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:45.001859  7503 raft_consensus.cc:3060] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:45.007093  7245 leader_election.cc:343] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.007633  7245 leader_election.cc:304] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.001507  7551 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.018558  7792 raft_consensus.cc:2749] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.028838  7551 tablet_bootstrap.cc:654] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.019958  7176 catalog_manager.cc:5697] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:45.033056  7468 leader_election.cc:343] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.040357  7554 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
W20260709 13:15:45.040652  7468 leader_election.cc:343] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.040900  7554 ts_tablet_manager.cc:1403] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.003s
I20260709 13:15:45.041102  7468 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:15:45.021255  7320 leader_election.cc:343] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.045756  7857 raft_consensus.cc:2749] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.050892  7550 tablet_bootstrap.cc:654] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.043376  7503 raft_consensus.cc:2468] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:45.054905  7554 raft_consensus.cc:359] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.055685  7554 raft_consensus.cc:385] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.056001  7554 raft_consensus.cc:740] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.070932  7554 consensus_queue.cc:260] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.073807  7554 ts_tablet_manager.cc:1434] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:15:45.075136  7554 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:45.078799  7843 raft_consensus.cc:493] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.079532  7843 raft_consensus.cc:515] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.080907  7554 tablet_bootstrap.cc:654] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.087046  7552 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:45.087654  7552 ts_tablet_manager.cc:1403] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.228s	user 0.017s	sys 0.007s
I20260709 13:15:45.097019  7552 raft_consensus.cc:359] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.099457  7552 raft_consensus.cc:385] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.099885  7552 raft_consensus.cc:740] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.121652  7551 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.122237  7551 ts_tablet_manager.cc:1403] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.121s	user 0.025s	sys 0.016s
I20260709 13:15:45.129400  7843 leader_election.cc:290] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.129181  7551 raft_consensus.cc:359] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.129972  7551 raft_consensus.cc:385] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.130304  7551 raft_consensus.cc:740] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.131069  7551 consensus_queue.cc:260] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.103390  7320 leader_election.cc:304] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:45.133702  7551 ts_tablet_manager.cc:1434] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:15:45.136251  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.136130  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.137275  7280 raft_consensus.cc:2468] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
W20260709 13:15:45.137936  7320 leader_election.cc:343] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.139067  7551 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.140254  7850 raft_consensus.cc:2804] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.144076  7850 raft_consensus.cc:697] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:45.145833  7551 tablet_bootstrap.cc:654] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.148171  7552 consensus_queue.cc:260] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.150449  7857 consensus_queue.cc:1048] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.158840  7850 consensus_queue.cc:237] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.163015  7552 ts_tablet_manager.cc:1434] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.001s
I20260709 13:15:45.163551  7320 leader_election.cc:304] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:45.164542  7843 raft_consensus.cc:2804] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.165091  7843 raft_consensus.cc:493] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.165560  7843 raft_consensus.cc:3060] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.172205  7552 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:45.174027  7792 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.174606  7792 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.156459  7554 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:45.176121  7554 ts_tablet_manager.cc:1403] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.000s
I20260709 13:15:45.176775  7792 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.177606  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:45.178278  7505 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:45.178817  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.179507  7245 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:45.180925  7355 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:45.181334  7792 raft_consensus.cc:2804] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.181876  7792 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.182564  7792 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.179459  7550 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:45.188442  7550 ts_tablet_manager.cc:1403] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.206s	user 0.024s	sys 0.008s
I20260709 13:15:45.204641  7177 catalog_manager.cc:5697] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.191386  7550 raft_consensus.cc:359] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.208395  7550 raft_consensus.cc:385] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.208762  7550 raft_consensus.cc:740] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.211002  7792 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.214653  7554 raft_consensus.cc:359] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.217973  7554 raft_consensus.cc:385] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.218339  7554 raft_consensus.cc:740] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.219319  7554 consensus_queue.cc:260] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.222402  7554 ts_tablet_manager.cc:1434] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:15:45.213245  7843 raft_consensus.cc:515] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.225591  7843 leader_election.cc:290] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.227125  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:45.227860  7277 raft_consensus.cc:3060] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.230317  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:45.230964  7503 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.231729  7792 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.232580  7551 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.233075  7554 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:45.233108  7551 ts_tablet_manager.cc:1403] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.041s
I20260709 13:15:45.234675  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:45.235294  7355 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.236143  7552 tablet_bootstrap.cc:654] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.236044  7551 raft_consensus.cc:359] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.236758  7551 raft_consensus.cc:385] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.237067  7551 raft_consensus.cc:740] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.237912  7551 consensus_queue.cc:260] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.238667  7503 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:45.240105  7551 ts_tablet_manager.cc:1434] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:45.241303  7245 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:45.242863  7844 raft_consensus.cc:2804] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.243342  7844 raft_consensus.cc:697] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:45.244181  7844 consensus_queue.cc:237] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.250469  7554 tablet_bootstrap.cc:654] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.250846  7841 consensus_queue.cc:1048] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.267002  7552 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:45.267557  7552 ts_tablet_manager.cc:1403] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.103s	user 0.016s	sys 0.005s
I20260709 13:15:45.268505  7551 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.270534  7552 raft_consensus.cc:359] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.271312  7552 raft_consensus.cc:385] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.272945  7355 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:45.274626  7552 raft_consensus.cc:740] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.275486  7552 consensus_queue.cc:260] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.276558  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:15:45.279474  7320 leader_election.cc:343] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.277356  7277 raft_consensus.cc:2468] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:45.279582  7552 ts_tablet_manager.cc:1434] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:15:45.281304  7320 leader_election.cc:304] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:45.282577  7843 raft_consensus.cc:2804] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.281888  7552 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:45.283517  7843 raft_consensus.cc:697] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:45.284590  7843 consensus_queue.cc:237] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.286510  7551 tablet_bootstrap.cc:654] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.281978  7177 catalog_manager.cc:5697] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.226624  7550 consensus_queue.cc:260] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.295956  7792 consensus_queue.cc:1048] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.298990  7551 tablet_bootstrap.cc:492] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.299619  7551 ts_tablet_manager.cc:1403] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.002s
I20260709 13:15:45.301061  7554 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:45.301608  7554 ts_tablet_manager.cc:1403] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.008s
I20260709 13:15:45.301885  7550 ts_tablet_manager.cc:1434] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.001s
I20260709 13:15:45.303120  7550 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:45.302834  7551 raft_consensus.cc:359] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.306001  7551 raft_consensus.cc:385] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.306352  7551 raft_consensus.cc:740] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.309137  7552 tablet_bootstrap.cc:654] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.310235  7857 raft_consensus.cc:493] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.310772  7857 raft_consensus.cc:515] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.305560  7554 raft_consensus.cc:359] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.310879  7551 consensus_queue.cc:260] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.311736  7554 raft_consensus.cc:385] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.312080  7554 raft_consensus.cc:740] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.313057  7857 leader_election.cc:290] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.312942  7554 consensus_queue.cc:260] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.315229  7554 ts_tablet_manager.cc:1434] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:15:45.316370  7554 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:45.319839  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.320520  7551 ts_tablet_manager.cc:1434] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 13:15:45.321902  7471 leader_election.cc:343] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.322578  7551 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.329779  7554 tablet_bootstrap.cc:654] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.331735  7551 tablet_bootstrap.cc:654] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.335820  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.336565  7277 raft_consensus.cc:2468] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:45.338486  7468 leader_election.cc:304] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.340752  7610 raft_consensus.cc:2804] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.341210  7610 raft_consensus.cc:493] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.341621  7610 raft_consensus.cc:3060] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.343868  7551 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.344477  7551 ts_tablet_manager.cc:1403] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.006s
I20260709 13:15:45.347743  7551 raft_consensus.cc:359] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.348688  7551 raft_consensus.cc:385] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.348975  7551 raft_consensus.cc:740] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.349864  7771 raft_consensus.cc:493] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.350108  7552 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:45.349773  7551 consensus_queue.cc:260] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.350730  7552 ts_tablet_manager.cc:1403] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.001s
I20260709 13:15:45.350576  7771 raft_consensus.cc:515] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.353644  7552 raft_consensus.cc:359] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.354343  7552 raft_consensus.cc:385] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.354635  7552 raft_consensus.cc:740] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.354682  7771 leader_election.cc:290] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.355724  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.356503  7355 raft_consensus.cc:2468] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:45.358493  7397 leader_election.cc:304] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:45.358711  7610 raft_consensus.cc:515] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.359280  7717 raft_consensus.cc:2804] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.359727  7717 raft_consensus.cc:493] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.360072  7717 raft_consensus.cc:3060] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.362519  7554 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:45.363073  7554 ts_tablet_manager.cc:1403] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.004s
I20260709 13:15:45.364238  7550 tablet_bootstrap.cc:654] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.355489  7552 consensus_queue.cc:260] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.359740  7551 ts_tablet_manager.cc:1434] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:15:45.378553  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:45.379274  7276 raft_consensus.cc:3060] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.390170  7610 leader_election.cc:290] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.371938  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.391422  7552 ts_tablet_manager.cc:1434] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:15:45.392246  7277 raft_consensus.cc:2468] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:45.393088  7552 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:45.395506  7554 raft_consensus.cc:359] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.396230  7554 raft_consensus.cc:385] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.396510  7554 raft_consensus.cc:740] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.393996  7551 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.397257  7792 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.397794  7792 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.398409  7550 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:45.398936  7550 ts_tablet_manager.cc:1403] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.096s	user 0.011s	sys 0.008s
I20260709 13:15:45.402612  7717 raft_consensus.cc:515] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.405809  7276 raft_consensus.cc:2468] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:45.411027  7717 leader_election.cc:290] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.409409  7554 consensus_queue.cc:260] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.411389  7550 raft_consensus.cc:359] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.412235  7550 raft_consensus.cc:385] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.408759  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:45.413705  7468 leader_election.cc:304] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:45.413463  7554 ts_tablet_manager.cc:1434] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.002s
I20260709 13:15:45.414512  7277 raft_consensus.cc:3060] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.402374  7178 catalog_manager.cc:5697] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.415336  7787 raft_consensus.cc:2804] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.415875  7787 raft_consensus.cc:697] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:45.416699  7787 consensus_queue.cc:237] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.421135  7703 raft_consensus.cc:493] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.421911  7554 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:45.421708  7703 raft_consensus.cc:515] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.422848  7277 raft_consensus.cc:2468] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:45.408182  7552 tablet_bootstrap.cc:654] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.424212  7703 leader_election.cc:290] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.426505  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:45.427204  7352 raft_consensus.cc:3060] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.427712  7851 raft_consensus.cc:493] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.428193  7851 raft_consensus.cc:515] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.414132  7550 raft_consensus.cc:740] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.413964  7551 tablet_bootstrap.cc:654] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.430541  7851 leader_election.cc:290] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.430740  7550 consensus_queue.cc:260] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.432998  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:45.433169  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.433646  7355 raft_consensus.cc:3060] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.433856  7550 ts_tablet_manager.cc:1434] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:15:45.435046  7550 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
W20260709 13:15:45.435501  7471 leader_election.cc:343] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.439940  7554 tablet_bootstrap.cc:654] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.440188  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.440887  7427 raft_consensus.cc:2468] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:45.442190  7320 leader_election.cc:304] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:45.443002  7851 raft_consensus.cc:2804] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.443288  7552 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:45.443379  7851 raft_consensus.cc:493] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.443744  7792 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:45.443979  7552 ts_tablet_manager.cc:1403] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.004s
I20260709 13:15:45.444872  7787 consensus_queue.cc:1048] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.444056  7851 raft_consensus.cc:3060] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.445639  7394 leader_election.cc:304] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:45.448159  7550 tablet_bootstrap.cc:654] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.450788  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:45.452374  7245 leader_election.cc:343] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.453539  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:45.454213  7505 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:45.454880  7552 raft_consensus.cc:359] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.455186  7851 raft_consensus.cc:515] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.455569  7552 raft_consensus.cc:385] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.455814  7245 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:45.456128  7552 raft_consensus.cc:740] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.456935  7844 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.457386  7844 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.457824  7844 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.458072  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:45.458714  7276 raft_consensus.cc:3060] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.461391  7851 leader_election.cc:290] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.461589  7551 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.462262  7551 ts_tablet_manager.cc:1403] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.005s
I20260709 13:15:45.465123  7551 raft_consensus.cc:359] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.465803  7551 raft_consensus.cc:385] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.466082  7551 raft_consensus.cc:740] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.466228  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:45.466753  7551 consensus_queue.cc:260] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.457020  7552 consensus_queue.cc:260] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.471369  7276 raft_consensus.cc:2468] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:45.467438  7429 raft_consensus.cc:3060] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.472779  7320 leader_election.cc:304] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:45.473677  7851 raft_consensus.cc:2804] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.474169  7851 raft_consensus.cc:697] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:45.474995  7851 consensus_queue.cc:237] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.480346  7717 raft_consensus.cc:2804] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.480931  7717 raft_consensus.cc:697] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:45.481727  7717 consensus_queue.cc:237] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.486203  7610 raft_consensus.cc:493] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.486836  7610 raft_consensus.cc:515] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.488150  7551 ts_tablet_manager.cc:1434] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:15:45.502596  7703 consensus_queue.cc:1048] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.510776  7552 ts_tablet_manager.cc:1434] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.066s	user 0.003s	sys 0.005s
I20260709 13:15:45.523967  7552 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:45.492729  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.527782  7551 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.522250  7178 catalog_manager.cc:5697] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.530066  7771 raft_consensus.cc:493] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.530601  7771 raft_consensus.cc:515] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.532280  7427 raft_consensus.cc:2468] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:45.517251  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.533985  7277 raft_consensus.cc:2376] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1. Re-sending same reply.
I20260709 13:15:45.535416  7610 leader_election.cc:290] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:45.537169  7717 raft_consensus.cc:493] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.537685  7717 raft_consensus.cc:515] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.541309  7551 tablet_bootstrap.cc:654] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.541981  7352 raft_consensus.cc:2468] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:45.533330  7552 tablet_bootstrap.cc:654] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.546124  7771 leader_election.cc:290] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:45.551048  7468 leader_election.cc:304] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.552103  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.550603  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.552776  7276 raft_consensus.cc:2468] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:45.553339  7277 raft_consensus.cc:2393] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:45.553715  7717 leader_election.cc:290] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:45.560806  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.521842  7844 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.562096  7468 leader_election.cc:304] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:45.562968  7841 raft_consensus.cc:493] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.568002  7787 raft_consensus.cc:2804] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.568567  7787 raft_consensus.cc:493] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.568192  7841 raft_consensus.cc:515] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:45.569036  7787 raft_consensus.cc:3060] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.569885  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:45.570591  7841 leader_election.cc:290] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.572175  7844 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:45.572901  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:45.573518  7771 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.574486  7248 leader_election.cc:343] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.574656  7771 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.576813  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:45.577473  7503 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.579316  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:45.580241  7277 raft_consensus.cc:2393] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:45.580098  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:45.577010  7771 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:45.585676  7394 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:45.566223  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.570758  7505 raft_consensus.cc:2393] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:45.587569  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:45.603433  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:45.606649  7468 leader_election.cc:343] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.607419  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.590559  7554 tablet_bootstrap.cc:492] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:45.600167  7397 leader_election.cc:304] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.552721  7610 raft_consensus.cc:2804] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.604943  7355 raft_consensus.cc:2468] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:45.611804  7610 raft_consensus.cc:493] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.612414  7610 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.599385  7176 catalog_manager.cc:5697] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.589279  7771 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.592360  7787 raft_consensus.cc:515] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.611385  7554 ts_tablet_manager.cc:1403] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.190s	user 0.018s	sys 0.035s
W20260709 13:15:45.617652  7245 leader_election.cc:343] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.617095  7717 raft_consensus.cc:3060] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.622107  7397 leader_election.cc:304] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.609041  7503 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:45.634042  7245 leader_election.cc:304] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:45.651891  7245 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:45.653364  7842 raft_consensus.cc:2749] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.655773  7792 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.656497  7792 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.658564  7844 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.659304  7844 raft_consensus.cc:697] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:45.660420  7691 raft_consensus.cc:3060] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.660413  7844 consensus_queue.cc:237] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.665952  7792 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:45.699927  7787 leader_election.cc:290] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:45.700165  7551 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:45.700719  7551 ts_tablet_manager.cc:1403] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.001s
I20260709 13:15:45.704425  7610 raft_consensus.cc:515] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.703485  7551 raft_consensus.cc:359] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.707302  7551 raft_consensus.cc:385] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:45.681839  7397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 61340.1us
I20260709 13:15:45.707548  7551 raft_consensus.cc:740] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.715406  7277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:45.716351  7277 raft_consensus.cc:3060] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.717211  7554 raft_consensus.cc:359] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.717959  7554 raft_consensus.cc:385] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.718216  7554 raft_consensus.cc:740] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.718912  7554 consensus_queue.cc:260] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.721355  7551 consensus_queue.cc:260] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.695325  7842 consensus_queue.cc:1048] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:15:45.726787  7610 leader_election.cc:290] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:45.724459  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:45.729920  7431 raft_consensus.cc:3060] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:45.746639  7887 log.cc:927] Time spent T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:15:45.748360  7550 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:45.748842  7550 ts_tablet_manager.cc:1403] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.314s	user 0.027s	sys 0.027s
I20260709 13:15:45.755044  7552 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:45.756642  7552 ts_tablet_manager.cc:1403] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.234s	user 0.020s	sys 0.057s
I20260709 13:15:45.758194  7554 ts_tablet_manager.cc:1434] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.129s	user 0.007s	sys 0.000s
W20260709 13:15:45.744930  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 93182.7us
I20260709 13:15:45.744421  7429 raft_consensus.cc:2468] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:45.769938  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:45.770591  7355 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.771629  7771 raft_consensus.cc:2749] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.772732  7554 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:45.788177  7550 raft_consensus.cc:359] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.788955  7550 raft_consensus.cc:385] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.789232  7550 raft_consensus.cc:740] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
W20260709 13:15:45.766402  7397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 52005.8us
I20260709 13:15:45.793370  7550 consensus_queue.cc:260] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.798285  7551 ts_tablet_manager.cc:1434] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.097s	user 0.017s	sys 0.025s
I20260709 13:15:45.808643  7550 ts_tablet_manager.cc:1434] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.002s
I20260709 13:15:45.817160  7691 raft_consensus.cc:2749] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.825858  7176 catalog_manager.cc:5697] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.828079  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:45.836251  7552 raft_consensus.cc:359] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.837049  7552 raft_consensus.cc:385] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.837302  7552 raft_consensus.cc:740] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.838074  7552 consensus_queue.cc:260] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:45.841753  7792 consensus_queue.cc:1048] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.847415  7550 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:45.854712  7431 raft_consensus.cc:2468] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:45.848971  7552 ts_tablet_manager.cc:1434] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.092s	user 0.002s	sys 0.004s
I20260709 13:15:45.856578  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:45.857270  7426 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.838014  7843 consensus_queue.cc:1048] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
W20260709 13:15:45.865029  7245 leader_election.cc:343] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.873256  7550 tablet_bootstrap.cc:654] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.874047  7851 consensus_queue.cc:1048] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:45.879204  7468 leader_election.cc:304] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:45.883499  7674 consensus_queue.cc:1048] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.885152  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:45.888731  7245 leader_election.cc:343] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.870716  7355 raft_consensus.cc:2468] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:45.893929  7471 leader_election.cc:304] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:45.894174  7717 raft_consensus.cc:2749] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.897197  7789 raft_consensus.cc:2804] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.897737  7789 raft_consensus.cc:697] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:45.898578  7789 consensus_queue.cc:237] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:45.889096  7245 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:45.890815  7430 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:45.881922  7610 raft_consensus.cc:2804] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.906333  7610 raft_consensus.cc:697] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:45.907222  7610 consensus_queue.cc:237] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:45.912645  7792 raft_consensus.cc:2749] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.917069  7787 consensus_queue.cc:1048] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.910560  7277 raft_consensus.cc:2468] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:45.953212  7554 tablet_bootstrap.cc:654] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.948527  7178 catalog_manager.cc:5697] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.970443  7550 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:45.971004  7550 ts_tablet_manager.cc:1403] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.160s	user 0.021s	sys 0.001s
I20260709 13:15:45.977177  7551 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:45.981293  7552 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:45.998028  7552 tablet_bootstrap.cc:654] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.011221  7850 consensus_queue.cc:1048] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.022167  7551 tablet_bootstrap.cc:654] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.024194  7550 raft_consensus.cc:359] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.025856  7550 raft_consensus.cc:385] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.026150  7550 raft_consensus.cc:740] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.020598  7426 raft_consensus.cc:2468] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:46.029225  7550 consensus_queue.cc:260] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.032073  7789 raft_consensus.cc:493] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.033797  7844 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.034353  7844 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.040688  7550 ts_tablet_manager.cc:1434] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 13:15:46.045267  7844 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:46.054118  7787 consensus_queue.cc:1048] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:46.060503  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:46.062786  7550 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.041407  7789 raft_consensus.cc:515] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.065289  7789 leader_election.cc:290] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:46.068470  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:46.069222  7504 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:46.070505  7245 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:46.071754  7844 raft_consensus.cc:2804] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.072158  7844 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.072536  7844 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.072757  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:46.073354  7842 consensus_queue.cc:1048] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.076145  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:46.078316  7471 leader_election.cc:343] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.079413  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:46.083375  7468 leader_election.cc:343] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.083961  7468 leader_election.cc:304] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:15:46.091455  7248 leader_election.cc:343] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.091645  7717 consensus_queue.cc:1048] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.095535  7176 catalog_manager.cc:5697] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.098225  7176 catalog_manager.cc:5697] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.106415  7857 raft_consensus.cc:2749] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.149389  7554 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.150081  7554 ts_tablet_manager.cc:1403] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.378s	user 0.015s	sys 0.007s
I20260709 13:15:46.152334  7850 consensus_queue.cc:1048] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.152908  7554 raft_consensus.cc:359] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.153602  7554 raft_consensus.cc:385] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.153858  7554 raft_consensus.cc:740] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.154477  7554 consensus_queue.cc:260] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.155930  7844 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.157912  7554 ts_tablet_manager.cc:1434] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:46.158914  7554 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.160135  7717 consensus_queue.cc:1048] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.161958  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:15:46.163743  7248 leader_election.cc:343] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.164654  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:46.165320  7504 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.174095  7844 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:46.180841  7550 tablet_bootstrap.cc:654] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.194674  7504 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:46.202651  7554 tablet_bootstrap.cc:654] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.184990  7850 raft_consensus.cc:493] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.211042  7850 raft_consensus.cc:515] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.212890  7674 consensus_queue.cc:1048] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.213843  7850 leader_election.cc:290] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:46.221024  7245 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:46.225162  7894 raft_consensus.cc:2804] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.225797  7894 raft_consensus.cc:697] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:46.231545  7552 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:46.232394  7552 ts_tablet_manager.cc:1403] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.289s	user 0.019s	sys 0.013s
I20260709 13:15:46.226913  7894 consensus_queue.cc:237] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.257407  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:46.269897  7320 leader_election.cc:343] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:46.272666  7918 log.cc:927] Time spent T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:15:46.272745  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:46.273741  7505 raft_consensus.cc:2468] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:46.282737  7320 leader_election.cc:304] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:46.285399  7851 raft_consensus.cc:2804] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.285811  7851 raft_consensus.cc:493] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.286139  7851 raft_consensus.cc:3060] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.302090  7551 tablet_bootstrap.cc:492] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:46.302690  7551 ts_tablet_manager.cc:1403] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.366s	user 0.031s	sys 0.018s
I20260709 13:15:46.305482  7551 raft_consensus.cc:359] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.306245  7551 raft_consensus.cc:385] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.306531  7551 raft_consensus.cc:740] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.307251  7551 consensus_queue.cc:260] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.313732  7551 ts_tablet_manager.cc:1434] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:15:46.314721  7551 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:46.314558  7552 raft_consensus.cc:359] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.319959  7552 raft_consensus.cc:385] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.320343  7552 raft_consensus.cc:740] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.321929  7551 tablet_bootstrap.cc:654] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.330374  7552 consensus_queue.cc:260] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.332459  7844 consensus_queue.cc:1048] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.336742  7552 ts_tablet_manager.cc:1434] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.000s
I20260709 13:15:46.345151  7552 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:46.339295  7851 raft_consensus.cc:515] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.350175  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:46.351002  7426 raft_consensus.cc:3060] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.353583  7787 consensus_queue.cc:1048] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.362401  7554 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.363018  7554 ts_tablet_manager.cc:1403] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.204s	user 0.034s	sys 0.022s
I20260709 13:15:46.369004  7550 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:46.369540  7550 ts_tablet_manager.cc:1403] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.318s	user 0.017s	sys 0.037s
I20260709 13:15:46.369837  7426 raft_consensus.cc:2468] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:46.371220  7554 raft_consensus.cc:359] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.372465  7554 raft_consensus.cc:385] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.372732  7554 raft_consensus.cc:740] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.372884  7320 leader_election.cc:304] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:46.373704  7843 raft_consensus.cc:2804] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.374246  7843 raft_consensus.cc:697] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:46.373394  7554 consensus_queue.cc:260] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.375815  7554 ts_tablet_manager.cc:1434] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.006s
I20260709 13:15:46.375370  7843 consensus_queue.cc:237] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.376820  7554 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.375430  7550 raft_consensus.cc:359] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.377393  7550 raft_consensus.cc:385] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.377671  7550 raft_consensus.cc:740] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.381217  7550 consensus_queue.cc:260] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.382633  7550 ts_tablet_manager.cc:1434] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:15:46.383595  7550 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.384775  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:46.385413  7505 raft_consensus.cc:3060] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.393640  7851 leader_election.cc:290] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:46.396131  7554 tablet_bootstrap.cc:654] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.394049  7550 tablet_bootstrap.cc:654] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.396083  7505 raft_consensus.cc:2468] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:46.431690  7552 tablet_bootstrap.cc:654] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.442200  7554 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.442732  7554 ts_tablet_manager.cc:1403] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.000s
I20260709 13:15:46.449033  7787 consensus_queue.cc:1048] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.450366  7554 raft_consensus.cc:359] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.451279  7554 raft_consensus.cc:385] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.451411  7843 raft_consensus.cc:493] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.451565  7554 raft_consensus.cc:740] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.451970  7843 raft_consensus.cc:515] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.452425  7554 consensus_queue.cc:260] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.455175  7554 ts_tablet_manager.cc:1434] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:15:46.456844  7554 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.468158  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:46.469017  7280 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:46.478036  7789 consensus_queue.cc:1048] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:15:46.484934  7551 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:46.485718  7551 ts_tablet_manager.cc:1403] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.171s	user 0.019s	sys 0.001s
I20260709 13:15:46.480090  7610 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.488168  7610 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.496296  7610 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:46.498334  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:46.499913  7468 leader_election.cc:343] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.501428  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:46.503669  7843 leader_election.cc:290] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:46.507262  7320 leader_election.cc:304] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:46.508322  7919 raft_consensus.cc:2804] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.508689  7919 raft_consensus.cc:493] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.508981  7919 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.512750  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:46.513468  7275 raft_consensus.cc:2468] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:46.535310  7176 catalog_manager.cc:5697] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.540442  7778 consensus_queue.cc:1048] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:15:46.541869  7552 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:46.547755  7554 tablet_bootstrap.cc:654] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.548025  7552 ts_tablet_manager.cc:1403] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.203s	user 0.024s	sys 0.012s
I20260709 13:15:46.539523  7468 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:46.544387  7919 raft_consensus.cc:515] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.554735  7552 raft_consensus.cc:359] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.555457  7552 raft_consensus.cc:385] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.555753  7552 raft_consensus.cc:740] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.556569  7552 consensus_queue.cc:260] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.559192  7552 ts_tablet_manager.cc:1434] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:15:46.560173  7552 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:46.561219  7550 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:46.562099  7550 ts_tablet_manager.cc:1403] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.179s	user 0.009s	sys 0.007s
I20260709 13:15:46.566892  7178 catalog_manager.cc:5697] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.568044  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:46.568691  7275 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.572762  7551 raft_consensus.cc:359] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.573457  7551 raft_consensus.cc:385] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.573699  7551 raft_consensus.cc:740] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.555730  7787 raft_consensus.cc:2804] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.575471  7787 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.575930  7787 raft_consensus.cc:3060] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.576938  7910 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.577456  7910 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.567351  7550 raft_consensus.cc:359] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.579010  7550 raft_consensus.cc:385] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.581037  7550 raft_consensus.cc:740] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.581694  7550 consensus_queue.cc:260] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.582471  7910 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:46.583770  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:46.587900  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:46.588742  7502 raft_consensus.cc:2393] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:46.593585  7550 ts_tablet_manager.cc:1434] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:15:46.594448  7320 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:46.594749  7550 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.595916  7843 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.600004  7551 consensus_queue.cc:260] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.602831  7551 ts_tablet_manager.cc:1434] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.117s	user 0.006s	sys 0.000s
I20260709 13:15:46.603811  7426 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:46.606438  7550 tablet_bootstrap.cc:654] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.608819  7919 leader_election.cc:290] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:46.613780  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:46.614647  7426 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.620244  7843 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.641110  7551 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:46.652603  7610 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.653252  7552 tablet_bootstrap.cc:654] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.653208  7610 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.658239  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:46.659744  7471 leader_election.cc:343] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.660602  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:46.663717  7610 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:15:46.665130  7468 leader_election.cc:343] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.665563  7468 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:46.666386  7426 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:46.667625  7320 leader_election.cc:304] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:46.667761  7778 raft_consensus.cc:2749] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.673043  7843 raft_consensus.cc:2804] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.632058  7275 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:46.677819  7843 raft_consensus.cc:697] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:46.628185  7787 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.681597  7843 consensus_queue.cc:237] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.682988  7787 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:46.687207  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:46.688069  7275 raft_consensus.cc:3060] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.691704  7554 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.691838  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:46.692240  7554 ts_tablet_manager.cc:1403] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.236s	user 0.015s	sys 0.012s
W20260709 13:15:46.693267  7468 leader_election.cc:343] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.695079  7554 raft_consensus.cc:359] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.695770  7554 raft_consensus.cc:385] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.691704  7551 tablet_bootstrap.cc:654] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.696704  7894 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.697249  7894 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.696028  7554 raft_consensus.cc:740] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.699517  7552 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:46.700348  7552 ts_tablet_manager.cc:1403] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.140s	user 0.032s	sys 0.004s
I20260709 13:15:46.700726  7554 consensus_queue.cc:260] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.704258  7554 ts_tablet_manager.cc:1434] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:15:46.705237  7554 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.708801  7275 raft_consensus.cc:2468] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:46.712226  7554 tablet_bootstrap.cc:654] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.699361  7894 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:46.712283  7468 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:46.714778  7787 raft_consensus.cc:2804] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.715230  7787 raft_consensus.cc:697] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:46.715938  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:46.716048  7787 consensus_queue.cc:237] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.719110  7176 catalog_manager.cc:5697] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.720336  7843 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:46.721263  7245 leader_election.cc:343] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.721184  7843 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.721979  7550 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:46.722134  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:46.722551  7550 ts_tablet_manager.cc:1403] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.128s	user 0.030s	sys 0.017s
I20260709 13:15:46.723433  7843 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:46.725666  7550 raft_consensus.cc:359] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:46.728159  7248 leader_election.cc:343] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.731017  7248 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:46.729180  7552 raft_consensus.cc:359] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.731987  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:46.732283  7552 raft_consensus.cc:385] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.732609  7552 raft_consensus.cc:740] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.732721  7504 raft_consensus.cc:2468] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:46.733259  7844 raft_consensus.cc:2749] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:46.733417  7552 consensus_queue.cc:260] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.734432  7320 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:46.735327  7910 raft_consensus.cc:2804] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.735706  7910 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.736022  7910 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.742174  7551 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:46.742774  7551 ts_tablet_manager.cc:1403] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.117s	user 0.008s	sys 0.004s
I20260709 13:15:46.729950  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:46.745340  7320 leader_election.cc:343] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.745635  7551 raft_consensus.cc:359] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.746367  7551 raft_consensus.cc:385] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.746713  7551 raft_consensus.cc:740] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.747336  7554 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.748298  7552 ts_tablet_manager.cc:1434] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:15:46.756209  7552 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:46.756758  7554 ts_tablet_manager.cc:1403] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.000s
I20260709 13:15:46.758404  7910 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.760900  7554 raft_consensus.cc:359] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.761683  7554 raft_consensus.cc:385] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.762007  7554 raft_consensus.cc:740] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.762461  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:46.763026  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:46.752275  7894 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.763773  7894 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:46.764249  7320 leader_election.cc:343] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.765928  7894 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:46.766865  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:46.766954  7176 catalog_manager.cc:5697] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:46.768435  7248 leader_election.cc:343] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.763180  7504 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.762813  7554 consensus_queue.cc:260] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.748821  7551 consensus_queue.cc:260] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.734508  7550 raft_consensus.cc:385] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.770326  7550 raft_consensus.cc:740] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.771026  7550 consensus_queue.cc:260] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.771538  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:46.772239  7505 raft_consensus.cc:2468] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:46.773257  7552 tablet_bootstrap.cc:654] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.773583  7245 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:46.775127  7792 raft_consensus.cc:2804] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:46.775501  7792 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.775753  7550 ts_tablet_manager.cc:1434] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.000s
I20260709 13:15:46.775831  7792 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.778671  7550 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.785022  7550 tablet_bootstrap.cc:654] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.786561  7894 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.769914  7910 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:46.783094  7792 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.799088  7850 consensus_queue.cc:1048] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.799568  7792 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:46.806331  7551 ts_tablet_manager.cc:1434] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.002s
I20260709 13:15:46.810700  7552 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:46.811215  7552 ts_tablet_manager.cc:1403] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.004s
I20260709 13:15:46.811169  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:46.812085  7505 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:46.812458  7554 ts_tablet_manager.cc:1434] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:15:46.813443  7554 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.815289  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:46.819775  7551 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
W20260709 13:15:46.819461  7248 leader_election.cc:343] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:46.826355  7505 raft_consensus.cc:2468] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:46.827653  7245 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:46.828709  7792 raft_consensus.cc:2804] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.828888  7554 tablet_bootstrap.cc:654] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.829403  7792 raft_consensus.cc:697] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:46.832716  7504 raft_consensus.cc:2468] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:46.834048  7320 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:46.835172  7851 raft_consensus.cc:2804] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:46.835712  7851 raft_consensus.cc:697] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:46.836741  7851 consensus_queue.cc:237] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.838914  7551 tablet_bootstrap.cc:654] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.830461  7792 consensus_queue.cc:237] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.848058  7552 raft_consensus.cc:359] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.848827  7552 raft_consensus.cc:385] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.849086  7552 raft_consensus.cc:740] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.849675  7552 consensus_queue.cc:260] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.859596  7844 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.851861  7552 ts_tablet_manager.cc:1434] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.002s
I20260709 13:15:46.862749  7552 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:46.868279  7550 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:46.868930  7550 ts_tablet_manager.cc:1403] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.002s
I20260709 13:15:46.872219  7550 raft_consensus.cc:359] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.872884  7550 raft_consensus.cc:385] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.873144  7550 raft_consensus.cc:740] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.877221  7550 consensus_queue.cc:260] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.882346  7552 tablet_bootstrap.cc:654] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.885213  7550 ts_tablet_manager.cc:1434] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.004s
I20260709 13:15:46.890082  7554 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:46.890272  7919 consensus_queue.cc:1048] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.890652  7554 ts_tablet_manager.cc:1403] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.011s
I20260709 13:15:46.893230  7550 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.896425  7551 tablet_bootstrap.cc:492] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:46.896970  7551 ts_tablet_manager.cc:1403] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.085s	user 0.006s	sys 0.005s
I20260709 13:15:46.899544  7550 tablet_bootstrap.cc:654] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.901005  7179 catalog_manager.cc:5697] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.902913  7551 raft_consensus.cc:359] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.906680  7550 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:46.906798  7554 raft_consensus.cc:359] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.903632  7551 raft_consensus.cc:385] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.907706  7550 ts_tablet_manager.cc:1403] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.002s
I20260709 13:15:46.907701  7554 raft_consensus.cc:385] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.910106  7554 raft_consensus.cc:740] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.910851  7554 consensus_queue.cc:260] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:46.912266  7550 raft_consensus.cc:359] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.913334  7550 raft_consensus.cc:385] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.913764  7550 raft_consensus.cc:740] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.914706  7550 consensus_queue.cc:260] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.908159  7551 raft_consensus.cc:740] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.917533  7551 consensus_queue.cc:260] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:46.917901  7550 ts_tablet_manager.cc:1434] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:46.919080  7550 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:46.924371  7551 ts_tablet_manager.cc:1434] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:15:46.925422  7551 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:46.933552  7180 catalog_manager.cc:5697] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.945148  7554 ts_tablet_manager.cc:1434] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:15:46.952265  7554 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:46.961431  7551 tablet_bootstrap.cc:654] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.963557  7792 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.965066  7792 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:46.974089  7554 tablet_bootstrap.cc:654] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:46.974243  7552 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:46.974795  7552 ts_tablet_manager.cc:1403] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.112s	user 0.034s	sys 0.019s
I20260709 13:15:46.977252  7552 raft_consensus.cc:359] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.977896  7552 raft_consensus.cc:385] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:46.978139  7552 raft_consensus.cc:740] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:46.978797  7552 consensus_queue.cc:260] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:46.980952  7552 ts_tablet_manager.cc:1434] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:15:46.981866  7552 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:46.985363  7792 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:46.987663  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:46.989620  7245 leader_election.cc:343] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.005069  7552 tablet_bootstrap.cc:654] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.009135  7910 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.009658  7910 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.007227  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:47.013525  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.014209  7505 raft_consensus.cc:2393] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
W20260709 13:15:47.014391  7248 leader_election.cc:343] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.015018  7248 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.016180  7844 raft_consensus.cc:2749] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.019136  7910 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.023506  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.032730  7320 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:47.034368  7910 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.046967  7910 raft_consensus.cc:2749] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.051944  7550 tablet_bootstrap.cc:654] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.054716  7551 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:47.055235  7551 ts_tablet_manager.cc:1403] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.130s	user 0.029s	sys 0.020s
I20260709 13:15:47.057955  7551 raft_consensus.cc:359] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.058626  7551 raft_consensus.cc:385] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.058887  7551 raft_consensus.cc:740] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.059557  7551 consensus_queue.cc:260] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.061928  7554 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:47.062186  7552 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:47.062579  7554 ts_tablet_manager.cc:1403] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.117s	user 0.018s	sys 0.024s
I20260709 13:15:47.062716  7552 ts_tablet_manager.cc:1403] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.007s
I20260709 13:15:47.065420  7554 raft_consensus.cc:359] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.066197  7554 raft_consensus.cc:385] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.065966  7552 raft_consensus.cc:359] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.066588  7554 raft_consensus.cc:740] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.066896  7552 raft_consensus.cc:385] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.067378  7552 raft_consensus.cc:740] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.067760  7554 consensus_queue.cc:260] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.068185  7552 consensus_queue.cc:260] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.069944  7554 ts_tablet_manager.cc:1434] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:47.071681  7554 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:47.077883  7554 tablet_bootstrap.cc:654] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.082590  7691 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.083122  7691 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.089496  7551 ts_tablet_manager.cc:1434] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.004s
I20260709 13:15:47.090363  7552 ts_tablet_manager.cc:1434] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:15:47.090574  7778 raft_consensus.cc:493] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.091117  7778 raft_consensus.cc:515] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.093377  7778 leader_election.cc:290] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.097070  7552 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:47.108592  7851 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.109244  7851 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.112620  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.114881  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.115437  7551 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:47.115729  7552 tablet_bootstrap.cc:654] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.117492  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.118131  7275 raft_consensus.cc:2468] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:47.123107  7554 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:47.123664  7554 ts_tablet_manager.cc:1403] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.030s
I20260709 13:15:47.125509  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.123162  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.126391  7426 raft_consensus.cc:2468] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
W20260709 13:15:47.128618  7320 leader_election.cc:343] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.115513  7504 raft_consensus.cc:2468] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:47.129498  7843 consensus_queue.cc:1048] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.131654  7320 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:47.132050  7691 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.112607  7851 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:47.133311  7851 raft_consensus.cc:2804] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.133388  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.133803  7851 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.134188  7851 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.134195  7275 raft_consensus.cc:2393] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:47.135514  7550 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:47.136072  7550 ts_tablet_manager.cc:1403] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.217s	user 0.014s	sys 0.000s
I20260709 13:15:47.131632  7468 leader_election.cc:304] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:47.140266  7778 raft_consensus.cc:2804] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.140628  7778 raft_consensus.cc:493] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.140931  7778 raft_consensus.cc:3060] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.141390  7551 tablet_bootstrap.cc:654] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.139449  7394 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:47.142946  7962 raft_consensus.cc:3060] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.144855  7843 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.145359  7843 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.139168  7550 raft_consensus.cc:359] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.146261  7550 raft_consensus.cc:385] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.146596  7550 raft_consensus.cc:740] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.148175  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.149044  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.145365  7851 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.152720  7275 raft_consensus.cc:2468] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
W20260709 13:15:47.152918  7320 leader_election.cc:343] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.149549  7843 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:47.154006  7550 consensus_queue.cc:260] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.154506  7320 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:47.157022  7552 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:47.157524  7552 ts_tablet_manager.cc:1403] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.061s	user 0.004s	sys 0.006s
I20260709 13:15:47.158931  7843 raft_consensus.cc:2804] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.159538  7843 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.160013  7843 raft_consensus.cc:3060] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.161211  7550 ts_tablet_manager.cc:1434] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:15:47.162619  7550 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:47.168934  7550 tablet_bootstrap.cc:654] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.150310  7778 raft_consensus.cc:515] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.171406  7778 leader_election.cc:290] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.172961  7851 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:47.126407  7554 raft_consensus.cc:359] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.174307  7552 raft_consensus.cc:359] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.174697  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:47.177326  7504 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.174697  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:47.180768  7894 raft_consensus.cc:493] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.181325  7894 raft_consensus.cc:515] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:47.184005  7320 leader_election.cc:343] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.185765  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.174569  7554 raft_consensus.cc:385] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.186396  7554 raft_consensus.cc:740] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.186467  7551 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
W20260709 13:15:47.187403  7245 leader_election.cc:343] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.188081  7551 ts_tablet_manager.cc:1403] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.000s
I20260709 13:15:47.190128  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:47.190827  7428 raft_consensus.cc:3060] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.192006  7550 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:47.189188  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:47.192731  7550 ts_tablet_manager.cc:1403] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.001s
I20260709 13:15:47.192760  7552 raft_consensus.cc:385] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.193272  7552 raft_consensus.cc:740] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.194516  7551 raft_consensus.cc:359] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.183399  7894 leader_election.cc:290] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:47.195214  7551 raft_consensus.cc:385] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.195536  7551 raft_consensus.cc:740] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
W20260709 13:15:47.196729  7248 leader_election.cc:343] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.197371  7248 leader_election.cc:304] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.198523  7894 raft_consensus.cc:2749] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.200382  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:47.201032  7275 raft_consensus.cc:3060] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.204074  7703 consensus_queue.cc:1048] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.206837  7552 consensus_queue.cc:260] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.209486  7552 ts_tablet_manager.cc:1434] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.052s	user 0.013s	sys 0.000s
I20260709 13:15:47.189452  7504 raft_consensus.cc:2468] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:47.187220  7554 consensus_queue.cc:260] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.229910  7552 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:47.227164  7320 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:47.248229  7910 raft_consensus.cc:2804] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.248723  7910 raft_consensus.cc:697] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:47.249292  7275 raft_consensus.cc:2468] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:47.249518  7910 consensus_queue.cc:237] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.256052  7851 consensus_queue.cc:1048] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.258041  7552 tablet_bootstrap.cc:654] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.259877  7468 leader_election.cc:304] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:47.231408  7428 raft_consensus.cc:2468] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:47.261742  7778 raft_consensus.cc:2804] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.262244  7778 raft_consensus.cc:697] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:47.256351  7850 consensus_queue.cc:1048] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:15:47.230103  7551 consensus_queue.cc:260] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.271941  7691 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.272414  7691 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.278239  7551 ts_tablet_manager.cc:1434] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.087s	user 0.014s	sys 0.000s
I20260709 13:15:47.279371  7551 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:47.282140  7691 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:47.282495  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.283111  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:47.283255  7276 raft_consensus.cc:2468] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:47.284546  7394 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:47.286072  7691 raft_consensus.cc:2804] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.286500  7691 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.286907  7691 raft_consensus.cc:3060] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.263166  7778 consensus_queue.cc:237] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.226249  7550 raft_consensus.cc:359] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.290664  7550 raft_consensus.cc:385] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.290957  7550 raft_consensus.cc:740] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.293136  7550 consensus_queue.cc:260] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.307178  7550 ts_tablet_manager.cc:1434] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.114s	user 0.004s	sys 0.008s
I20260709 13:15:47.308316  7550 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:47.312150  7428 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.322821  7787 raft_consensus.cc:493] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.323455  7787 raft_consensus.cc:515] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.326277  7787 leader_election.cc:290] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:15:47.240310  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:47.327354  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:47.330541  7468 leader_election.cc:343] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.331127  7850 consensus_queue.cc:1048] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.338964  7961 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.339519  7961 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.341558  7961 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:47.342711  7551 tablet_bootstrap.cc:654] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.343073  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:47.340728  7178 catalog_manager.cc:5697] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:47.344455  7894 consensus_queue.cc:1048] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.345599  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.349431  7397 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.350739  7961 raft_consensus.cc:2749] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:47.360325  7691 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.364392  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:47.366591  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:47.367617  7275 raft_consensus.cc:3060] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.227828  7554 ts_tablet_manager.cc:1434] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.000s
I20260709 13:15:47.373173  7554 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:47.327811  7968 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.376405  7968 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.381273  7554 tablet_bootstrap.cc:654] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.327699  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:47.384549  7471 leader_election.cc:343] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.233922  7962 raft_consensus.cc:2749] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.385082  7471 leader_election.cc:304] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.387887  7968 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:47.373584  7691 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:47.388993  7550 tablet_bootstrap.cc:654] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.391582  7778 raft_consensus.cc:2749] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.394304  7778 consensus_queue.cc:1048] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.366998  7551 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:47.397665  7551 ts_tablet_manager.cc:1403] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.005s
I20260709 13:15:47.389415  7792 consensus_queue.cc:1048] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:47.396842  7275 raft_consensus.cc:2468] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:47.403716  7394 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.410077  7691 raft_consensus.cc:2804] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.411136  7691 raft_consensus.cc:697] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:47.245306  7843 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.412637  7355 raft_consensus.cc:2393] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:47.414150  7843 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:47.412638  7691 consensus_queue.cc:237] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.402494  7551 raft_consensus.cc:359] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.438861  7551 raft_consensus.cc:385] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.440424  7551 raft_consensus.cc:740] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.443071  7551 consensus_queue.cc:260] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.455363  7551 ts_tablet_manager.cc:1434] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.001s
I20260709 13:15:47.457553  7551 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:47.464838  7551 tablet_bootstrap.cc:654] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:47.470168  7320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 139789us
I20260709 13:15:47.474534  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.464591  7180 catalog_manager.cc:5697] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:47.485340  7787 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.485916  7787 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:47.493552  7320 leader_election.cc:343] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.503424  7550 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:47.504258  7550 ts_tablet_manager.cc:1403] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.196s	user 0.013s	sys 0.008s
I20260709 13:15:47.505146  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.488000  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:47.492758  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.510308  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:47.515503  7276 raft_consensus.cc:2393] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 1: Already voted for candidate 5270a004873440ec9a5e8106b63a6a85 in this term.
I20260709 13:15:47.517429  7552 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:47.518025  7552 ts_tablet_manager.cc:1403] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.308s	user 0.052s	sys 0.128s
I20260709 13:15:47.521128  7275 raft_consensus.cc:2468] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:47.509280  7787 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.521375  7429 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:47.539218  7841 raft_consensus.cc:493] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.549182  7176 catalog_manager.cc:5697] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:47.541261  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.555550  7550 raft_consensus.cc:359] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.560663  7554 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:47.561396  7554 ts_tablet_manager.cc:1403] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.188s	user 0.018s	sys 0.012s
I20260709 13:15:47.560380  7468 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:47.566978  7550 raft_consensus.cc:385] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.567404  7550 raft_consensus.cc:740] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.559461  7894 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.573354  7320 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:47.573966  7552 raft_consensus.cc:359] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.588653  7552 raft_consensus.cc:385] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.589098  7552 raft_consensus.cc:740] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.591650  7552 consensus_queue.cc:260] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.590766  7320 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:47.592052  7550 consensus_queue.cc:260] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.556751  7841 raft_consensus.cc:515] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.616565  7552 ts_tablet_manager.cc:1434] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.098s	user 0.011s	sys 0.000s
I20260709 13:15:47.628243  7550 ts_tablet_manager.cc:1434] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.115s	user 0.008s	sys 0.000s
I20260709 13:15:47.593905  7974 raft_consensus.cc:2804] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.596787  7968 raft_consensus.cc:2749] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.638202  7550 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
W20260709 13:15:47.641605  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 81303.6us
I20260709 13:15:47.593215  7894 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.638933  7974 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.649746  7990 raft_consensus.cc:2804] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.664309  7551 tablet_bootstrap.cc:492] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:47.664886  7551 ts_tablet_manager.cc:1403] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.208s	user 0.015s	sys 0.022s
I20260709 13:15:47.657516  7990 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.676275  7990 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:47.659469  7986 log.cc:927] Time spent T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:15:47.656677  7974 raft_consensus.cc:3060] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.662735  7841 leader_election.cc:290] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:47.678184  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:47.672848  7275 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:47.715772  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.719847  7554 raft_consensus.cc:359] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.717417  7552 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:47.720618  7554 raft_consensus.cc:385] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.720917  7554 raft_consensus.cc:740] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.721590  7554 consensus_queue.cc:260] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.730773  7552 tablet_bootstrap.cc:654] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.740931  7894 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:47.727029  7551 raft_consensus.cc:359] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.745455  7551 raft_consensus.cc:385] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.745600  7550 tablet_bootstrap.cc:654] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.745913  7551 raft_consensus.cc:740] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.746765  7551 consensus_queue.cc:260] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:47.776430  7551 ts_tablet_manager.cc:1434] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.002s
I20260709 13:15:47.779778  7554 ts_tablet_manager.cc:1434] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.218s	user 0.008s	sys 0.000s
W20260709 13:15:47.774751  7471 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 120236us
W20260709 13:15:47.786726  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 94774.9us
W20260709 13:15:47.792760  7248 leader_election.cc:343] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:47.840447  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 47493us
W20260709 13:15:47.841156  7248 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 47735.5us
W20260709 13:15:47.843113  7245 leader_election.cc:343] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:47.844801  7245 leader_election.cc:304] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:47.845868  7990 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.848024  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:47.848717  7430 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:47.782569  7974 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:47.866868  7554 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:47.869930  7245 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:47.877422  7792 raft_consensus.cc:2804] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.877823  7792 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.878157  7792 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.882751  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.885196  7504 raft_consensus.cc:2393] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b2b311085b64300925884dec9a28c7d in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:47.866955  7844 raft_consensus.cc:493] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.865549  8000 raft_consensus.cc:2749] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:47.901109  7844 raft_consensus.cc:515] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.924300  7844 leader_election.cc:290] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:47.925305  7787 raft_consensus.cc:493] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.925845  7787 raft_consensus.cc:515] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.927529  7987 consensus_queue.cc:1048] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.930197  7551 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:47.933415  7552 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: No bootstrap required, opened a new log
I20260709 13:15:47.933939  7552 ts_tablet_manager.cc:1403] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.239s	user 0.023s	sys 0.020s
I20260709 13:15:47.937939  7551 tablet_bootstrap.cc:654] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.938464  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:47.939203  7278 raft_consensus.cc:2468] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:47.941866  7554 tablet_bootstrap.cc:654] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.942158  7552 raft_consensus.cc:359] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.942919  7552 raft_consensus.cc:385] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.943207  7552 raft_consensus.cc:740] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.944208  7990 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:47.945312  7552 consensus_queue.cc:260] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:47.955267  7430 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.963235  7550 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:47.963821  7550 ts_tablet_manager.cc:1403] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.330s	user 0.017s	sys 0.002s
I20260709 13:15:47.969148  7468 leader_election.cc:304] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:47.969538  7991 consensus_queue.cc:1048] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:47.970727  7999 raft_consensus.cc:2804] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:47.970817  7787 leader_election.cc:290] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:47.971210  7999 raft_consensus.cc:493] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.966492  7550 raft_consensus.cc:359] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:47.972108  7550 raft_consensus.cc:385] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.972381  7550 raft_consensus.cc:740] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.947506  7552 ts_tablet_manager.cc:1434] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
W20260709 13:15:47.975620  7248 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 72674.5us
I20260709 13:15:47.976792  7550 consensus_queue.cc:260] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:47.981859  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 76404.4us
I20260709 13:15:47.984452  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:47.991220  7550 ts_tablet_manager.cc:1434] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:15:47.975047  7999 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.001523  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:48.007513  7550 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: Bootstrap starting.
I20260709 13:15:48.002216  7427 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.020771  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.038836  7691 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.039417  7691 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:15:48.041316  7245 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 44467.9us
I20260709 13:15:47.996241  7990 consensus_queue.cc:1048] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:15:48.042562  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.052510  7282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:48.053251  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:48.059350  7282 raft_consensus.cc:3060] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.067750  7843 consensus_queue.cc:1048] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:15:48.070710  7554 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
W20260709 13:15:48.072182  7471 leader_election.cc:343] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.085788  7554 ts_tablet_manager.cc:1403] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.219s	user 0.011s	sys 0.017s
I20260709 13:15:48.078168  7987 raft_consensus.cc:493] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.087021  7987 raft_consensus.cc:515] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:48.088434  7427 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:48.088887  7554 raft_consensus.cc:359] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.089613  7554 raft_consensus.cc:385] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.089701  7468 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
W20260709 13:15:48.079695  7248 leader_election.cc:343] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.089210  7987 leader_election.cc:290] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:48.092409  7554 raft_consensus.cc:740] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.097621  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:48.098312  7501 raft_consensus.cc:2468] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 0.
I20260709 13:15:48.099541  7320 leader_election.cc:304] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:48.099754  7787 raft_consensus.cc:2804] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.100255  7787 raft_consensus.cc:697] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:48.100430  7995 raft_consensus.cc:2804] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.100833  7995 raft_consensus.cc:493] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.101150  7995 raft_consensus.cc:3060] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.108611  7691 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:48.109809  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:48.112603  7554 consensus_queue.cc:260] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.114281  7554 ts_tablet_manager.cc:1434] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.000s
I20260709 13:15:48.101168  7787 consensus_queue.cc:237] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.115298  7554 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:48.116106  7974 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:48.119344  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.123713  7995 raft_consensus.cc:515] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:48.126415  7792 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.130889  7994 consensus_queue.cc:1048] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:48.132085  7792 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:48.133399  7995 leader_election.cc:290] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:48.133752  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:48.134649  7426 raft_consensus.cc:2393] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2b2b311085b64300925884dec9a28c7d in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:48.135049  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:48.135634  7505 raft_consensus.cc:3060] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.136672  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:48.137209  7278 raft_consensus.cc:2393] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in current term 1: Already voted for candidate 2b2b311085b64300925884dec9a28c7d in this term.
I20260709 13:15:48.138180  7245 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:48.141966  7894 raft_consensus.cc:2749] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.155462  7394 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:48.158049  7505 raft_consensus.cc:2468] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:48.160636  7320 leader_election.cc:304] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:48.165812  7999 raft_consensus.cc:515] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.161553  7282 raft_consensus.cc:2468] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:48.168902  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:48.169005  7999 leader_election.cc:290] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.169975  7504 raft_consensus.cc:2393] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b2b311085b64300925884dec9a28c7d in current term 1: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:48.170173  7320 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:48.157658  7994 raft_consensus.cc:2749] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:48.176901  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:48.177522  7551 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:48.178083  7551 ts_tablet_manager.cc:1403] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.248s	user 0.019s	sys 0.025s
I20260709 13:15:48.177899  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:48.182482  7850 raft_consensus.cc:2804] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.183082  7850 raft_consensus.cc:697] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:48.184294  7850 consensus_queue.cc:237] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.186659  7554 tablet_bootstrap.cc:654] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.187623  7245 leader_election.cc:304] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:48.186307  7551 raft_consensus.cc:359] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.189383  7551 raft_consensus.cc:385] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.189670  7551 raft_consensus.cc:740] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.191315  7551 consensus_queue.cc:260] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.198148  7994 consensus_queue.cc:1048] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:15:48.163831  7995 raft_consensus.cc:2804] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.194075  7276 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:48.200554  7320 leader_election.cc:343] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.215888  7551 ts_tablet_manager.cc:1434] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
W20260709 13:15:48.230773  7320 leader_election.cc:343] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.212416  7995 raft_consensus.cc:697] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:48.235594  7551 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:48.239132  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:48.239828  7357 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.234066  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8142e76af31447acb3186673c2a9934a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:48.249164  7995 consensus_queue.cc:237] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:48.254717  7179 catalog_manager.cc:5697] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.281638  7550 tablet_bootstrap.cc:654] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.296351  7551 tablet_bootstrap.cc:654] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:48.307986  7320 leader_election.cc:343] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.309242  7357 raft_consensus.cc:2468] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:48.312855  7471 leader_election.cc:304] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:48.313795  7999 raft_consensus.cc:2804] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.314316  7999 raft_consensus.cc:697] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:48.315161  7999 consensus_queue.cc:237] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.328361  7717 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.336385  7717 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.340979  7278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:48.338485  7717 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.341871  7278 raft_consensus.cc:2393] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 1: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
I20260709 13:15:48.346738  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.354324  7397 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:48.356949  7994 raft_consensus.cc:3060] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.344112  7276 raft_consensus.cc:2468] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:48.393884  8000 raft_consensus.cc:2749] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.402173  7554 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:48.431613  7554 ts_tablet_manager.cc:1403] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.316s	user 0.018s	sys 0.004s
I20260709 13:15:48.452795  7792 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.453279  7792 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.455124  7792 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
W20260709 13:15:48.459137  7228 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:48.459457  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:48.460798  7245 leader_election.cc:343] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.462111  7991 raft_consensus.cc:493] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.462354  7550 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: No bootstrap required, opened a new log
I20260709 13:15:48.462893  7550 ts_tablet_manager.cc:1403] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent bootstrapping tablet: real 0.463s	user 0.049s	sys 0.046s
I20260709 13:15:48.462657  7991 raft_consensus.cc:515] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.466343  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:48.471165  7554 raft_consensus.cc:359] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.472025  7554 raft_consensus.cc:385] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.472291  7554 raft_consensus.cc:740] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.473023  7554 consensus_queue.cc:260] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.475865  7994 raft_consensus.cc:2749] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.477919  7550 raft_consensus.cc:359] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.478610  7550 raft_consensus.cc:385] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.479674  7554 ts_tablet_manager.cc:1434] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:15:48.480700  7554 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: Bootstrap starting.
I20260709 13:15:48.478883  7550 raft_consensus.cc:740] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.475879  7178 catalog_manager.cc:5697] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.490099  7178 catalog_manager.cc:5697] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.494254  7550 consensus_queue.cc:260] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.495262  7505 raft_consensus.cc:2468] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 0.
I20260709 13:15:48.497259  7245 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:48.498026  7792 raft_consensus.cc:2804] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.498410  7792 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.498754  7792 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.501319  7550 ts_tablet_manager.cc:1434] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.005s
I20260709 13:15:48.522480  7792 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.522897  7991 leader_election.cc:290] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.531831  7275 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:48.532636  7275 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
I20260709 13:15:48.533089  7554 tablet_bootstrap.cc:654] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.534021  7792 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:48.534471  7551 tablet_bootstrap.cc:492] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:48.534581  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:48.535111  7551 ts_tablet_manager.cc:1403] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.307s	user 0.032s	sys 0.008s
I20260709 13:15:48.535667  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:48.536254  7505 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.536201  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.537063  7357 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 0.
W20260709 13:15:48.538405  7245 leader_election.cc:343] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.542058  7468 leader_election.cc:304] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:48.543182  7787 raft_consensus.cc:2804] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.543552  7787 raft_consensus.cc:493] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.543864  7787 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.546315  7551 raft_consensus.cc:359] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:48.547014  7551 raft_consensus.cc:385] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.547269  7551 raft_consensus.cc:740] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.551978  7551 consensus_queue.cc:260] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:48.553537  7787 raft_consensus.cc:515] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.554112  7551 ts_tablet_manager.cc:1434] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:15:48.555027  7551 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:48.557986  7505 raft_consensus.cc:2468] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:15:48.562155  7245 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:48.560990  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:48.562963  7357 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.563136  7792 raft_consensus.cc:2804] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.563692  7787 leader_election.cc:290] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.563871  7792 raft_consensus.cc:697] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:15:48.564697  7792 consensus_queue.cc:237] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.567032  7551 tablet_bootstrap.cc:654] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.570220  7357 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:48.574748  7179 catalog_manager.cc:5697] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.560509  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:48.581950  7280 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.588552  7554 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: No bootstrap required, opened a new log
I20260709 13:15:48.589653  7554 ts_tablet_manager.cc:1403] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent bootstrapping tablet: real 0.109s	user 0.008s	sys 0.010s
I20260709 13:15:48.590586  7280 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:48.591073  7919 consensus_queue.cc:1048] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.591941  7468 leader_election.cc:304] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:15:48.593051  7787 raft_consensus.cc:2804] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.593634  7787 raft_consensus.cc:697] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:48.592924  7554 raft_consensus.cc:359] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.594748  7554 raft_consensus.cc:385] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.594380  7787 consensus_queue.cc:237] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:48.595901  7554 raft_consensus.cc:740] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.607090  7554 consensus_queue.cc:260] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.610414  7554 ts_tablet_manager.cc:1434] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:15:48.616072  7551 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:48.616618  7551 ts_tablet_manager.cc:1403] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.010s
I20260709 13:15:48.619349  7551 raft_consensus.cc:359] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.620031  7551 raft_consensus.cc:385] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.620294  7551 raft_consensus.cc:740] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.621052  7551 consensus_queue.cc:260] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.624702  7551 ts_tablet_manager.cc:1434] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:48.625725  7551 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: Bootstrap starting.
I20260709 13:15:48.629594  7910 consensus_queue.cc:1048] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.634421  7551 tablet_bootstrap.cc:654] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.638895  7178 catalog_manager.cc:5697] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 1 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:48.654913  7180 catalog_manager.cc:5697] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 1 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.672658  7990 consensus_queue.cc:1048] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.675685  7431 raft_consensus.cc:3060] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.693077  7910 consensus_queue.cc:1048] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.694309  7551 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: No bootstrap required, opened a new log
I20260709 13:15:48.694890  7551 ts_tablet_manager.cc:1403] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: Time spent bootstrapping tablet: real 0.069s	user 0.020s	sys 0.030s
I20260709 13:15:48.698032  7551 raft_consensus.cc:359] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.698732  7551 raft_consensus.cc:385] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.699106  7551 raft_consensus.cc:740] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.699754  7551 consensus_queue.cc:260] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:48.701772  7551 ts_tablet_manager.cc:1434] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:15:48.734511  7987 consensus_queue.cc:1048] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.736948  7990 consensus_queue.cc:1048] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.807649  7990 consensus_queue.cc:1048] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.815724  7999 consensus_queue.cc:1048] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:48.933833  7994 raft_consensus.cc:493] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.934362  7994 raft_consensus.cc:515] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.936508  7994 leader_election.cc:290] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.937183  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:48.937819  7280 raft_consensus.cc:2468] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:48.937492  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:48.938437  7355 raft_consensus.cc:2468] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 0.
I20260709 13:15:48.938797  7394 leader_election.cc:304] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:48.939589  7994 raft_consensus.cc:2804] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.940012  7994 raft_consensus.cc:493] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.940342  7994 raft_consensus.cc:3060] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.945940  7994 raft_consensus.cc:515] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.948393  7994 leader_election.cc:290] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:48.948890  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:48.949280  7280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:48.949505  7357 raft_consensus.cc:3060] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.949929  7280 raft_consensus.cc:3060] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.957137  7357 raft_consensus.cc:2468] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:48.958298  7397 leader_election.cc:304] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:48.958381  7280 raft_consensus.cc:2468] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:48.959817  7994 raft_consensus.cc:2804] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.960366  7994 raft_consensus.cc:697] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:48.961221  7994 consensus_queue.cc:237] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:48.972216  7178 catalog_manager.cc:5697] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.012707  8000 consensus_queue.cc:1048] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.021296  7431 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.030155  8000 consensus_queue.cc:1048] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.031880  7994 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.032447  7994 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:49.034523  7994 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:49.035859  7276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:49.035838  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:49.041263  7394 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:49.042054  7962 raft_consensus.cc:2749] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:49.092640  7999 consensus_queue.cc:1048] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.104813  7787 consensus_queue.cc:1048] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.389523  7962 consensus_queue.cc:1048] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.400207  7691 consensus_queue.cc:1048] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:50.481302 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
W20260709 13:15:50.544305  7468 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 52 similar messages]
I20260709 13:15:50.558133 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:50.563247 32224 tablet_replica.cc:333] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.564301 32224 raft_consensus.cc:2243] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.564844 32224 raft_consensus.cc:2272] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.567365 32224 tablet_replica.cc:333] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.568253 32224 raft_consensus.cc:2243] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:50.571043  7468 consensus_peers.cc:597] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.572139  7468 consensus_peers.cc:597] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.572768  7468 consensus_peers.cc:597] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.579484  7468 consensus_peers.cc:597] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.580315  7468 consensus_peers.cc:597] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.568811 32224 raft_consensus.cc:2272] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.585974 32224 tablet_replica.cc:333] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.587365 32224 raft_consensus.cc:2243] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:15:50.588550  7320 consensus_peers.cc:597] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.588575 32224 raft_consensus.cc:2272] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.591758 32224 tablet_replica.cc:333] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.592700 32224 raft_consensus.cc:2243] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.593397 32224 raft_consensus.cc:2272] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.600234 32224 tablet_replica.cc:333] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.601372 32224 raft_consensus.cc:2243] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.602505 32224 raft_consensus.cc:2272] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.607779  7394 consensus_peers.cc:597] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.616897 32224 tablet_replica.cc:333] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.617909 32224 raft_consensus.cc:2243] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.620709 32224 raft_consensus.cc:2272] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.624607 32224 tablet_replica.cc:333] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.625474 32224 raft_consensus.cc:2243] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.626408 32224 raft_consensus.cc:2272] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.628326 32224 tablet_replica.cc:333] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.629117 32224 raft_consensus.cc:2243] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.629592 32224 raft_consensus.cc:2272] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.631928 32224 tablet_replica.cc:333] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.632690 32224 raft_consensus.cc:2243] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.633206 32224 raft_consensus.cc:2272] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.635110 32224 tablet_replica.cc:333] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.635738 32224 raft_consensus.cc:2243] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.636476 32224 raft_consensus.cc:2272] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.639039 32224 tablet_replica.cc:333] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.639788 32224 raft_consensus.cc:2243] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.640288 32224 raft_consensus.cc:2272] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.642330 32224 tablet_replica.cc:333] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.643113 32224 raft_consensus.cc:2243] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.643779 32224 raft_consensus.cc:2272] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.645891 32224 tablet_replica.cc:333] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.646706 32224 raft_consensus.cc:2243] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.647583 32224 raft_consensus.cc:2272] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.649550 32224 tablet_replica.cc:333] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.650148 32224 raft_consensus.cc:2243] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.650558 32224 raft_consensus.cc:2272] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.652454 32224 tablet_replica.cc:333] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.653023 32224 raft_consensus.cc:2243] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.653416 32224 raft_consensus.cc:2272] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.655067 32224 tablet_replica.cc:333] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.655596 32224 raft_consensus.cc:2243] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.656073 32224 raft_consensus.cc:2272] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.657930 32224 tablet_replica.cc:333] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.658481 32224 raft_consensus.cc:2243] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.659228 32224 raft_consensus.cc:2272] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.661350 32224 tablet_replica.cc:333] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.662163 32224 raft_consensus.cc:2243] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.662719 32224 raft_consensus.cc:2272] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.664916 32224 tablet_replica.cc:333] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.665525 32224 raft_consensus.cc:2243] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.666267 32224 raft_consensus.cc:2272] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.668318 32224 tablet_replica.cc:333] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.668852 32224 raft_consensus.cc:2243] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.669782 32224 raft_consensus.cc:2272] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.672071 32224 tablet_replica.cc:333] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.672780 32224 raft_consensus.cc:2243] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.673820 32224 raft_consensus.cc:2272] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.676223 32224 tablet_replica.cc:333] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.676812 32224 raft_consensus.cc:2243] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.677238 32224 raft_consensus.cc:2272] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.679024 32224 tablet_replica.cc:333] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.679561 32224 raft_consensus.cc:2243] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.680052 32224 raft_consensus.cc:2272] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.681903 32224 tablet_replica.cc:333] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.682614 32224 raft_consensus.cc:2243] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.683084 32224 raft_consensus.cc:2272] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.685322 32224 tablet_replica.cc:333] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.685776 32224 raft_consensus.cc:2243] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.686110 32224 raft_consensus.cc:2272] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.688118 32224 tablet_replica.cc:333] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.688584 32224 raft_consensus.cc:2243] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.689258 32224 raft_consensus.cc:2272] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.691116 32224 tablet_replica.cc:333] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.691695 32224 raft_consensus.cc:2243] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.692528 32224 raft_consensus.cc:2272] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.694721 32224 tablet_replica.cc:333] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.695328 32224 raft_consensus.cc:2243] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.695792 32224 raft_consensus.cc:2272] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.698346 32224 tablet_replica.cc:333] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
W20260709 13:15:50.698956  7394 consensus_peers.cc:597] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.699131 32224 raft_consensus.cc:2243] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.700258 32224 raft_consensus.cc:2272] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.700639  7468 consensus_peers.cc:597] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.702890 32224 tablet_replica.cc:333] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.703589 32224 raft_consensus.cc:2243] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.704384 32224 raft_consensus.cc:2272] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.706329 32224 tablet_replica.cc:333] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.706853 32224 raft_consensus.cc:2243] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.707273 32224 raft_consensus.cc:2272] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.709653 32224 tablet_replica.cc:333] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.710276 32224 raft_consensus.cc:2243] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.710731 32224 raft_consensus.cc:2272] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.712893 32224 tablet_replica.cc:333] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.713467 32224 raft_consensus.cc:2243] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.713835 32224 raft_consensus.cc:2272] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.716226 32224 tablet_replica.cc:333] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.716791 32224 raft_consensus.cc:2243] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.717298 32224 raft_consensus.cc:2272] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.719379 32224 tablet_replica.cc:333] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.719945 32224 raft_consensus.cc:2243] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.720811 32224 raft_consensus.cc:2272] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.723040 32224 tablet_replica.cc:333] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.723668 32224 raft_consensus.cc:2243] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.724118 32224 raft_consensus.cc:2272] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.724954  7468 consensus_peers.cc:597] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.726203 32224 tablet_replica.cc:333] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.727063 32224 raft_consensus.cc:2243] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.727558 32224 raft_consensus.cc:2272] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.729539 32224 tablet_replica.cc:333] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.730103 32224 raft_consensus.cc:2243] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.730500 32224 raft_consensus.cc:2272] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.732524 32224 tablet_replica.cc:333] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.733168 32224 raft_consensus.cc:2243] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.733613 32224 raft_consensus.cc:2272] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.734508  7468 consensus_peers.cc:597] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.735306 32224 tablet_replica.cc:333] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.735954 32224 raft_consensus.cc:2243] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.736699 32224 raft_consensus.cc:2272] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.738816 32224 tablet_replica.cc:333] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.739341 32224 raft_consensus.cc:2243] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.739832 32224 raft_consensus.cc:2272] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.741479  7468 consensus_peers.cc:597] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.742112 32224 tablet_replica.cc:333] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.742657 32224 raft_consensus.cc:2243] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.743028 32224 raft_consensus.cc:2272] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.745040 32224 tablet_replica.cc:333] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.745483 32224 raft_consensus.cc:2243] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.745814 32224 raft_consensus.cc:2272] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.747540 32224 tablet_replica.cc:333] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.748070 32224 raft_consensus.cc:2243] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:50.748409 32224 raft_consensus.cc:2272] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:50.750286 32224 tablet_replica.cc:333] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: stopping tablet replica
I20260709 13:15:50.750698 32224 raft_consensus.cc:2243] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:50.751433 32224 raft_consensus.cc:2272] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:50.783473  7394 consensus_peers.cc:597] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.795228  7468 consensus_peers.cc:597] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.813946  7320 consensus_peers.cc:597] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.815922  7320 consensus_peers.cc:597] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.821535  7320 consensus_peers.cc:597] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.830535 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
W20260709 13:15:50.841604  7320 consensus_peers.cc:597] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.861100  7468 consensus_peers.cc:597] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.861605  7468 consensus_peers.cc:597] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:50.889840  7320 consensus_peers.cc:597] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.905506 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:50.913776  8057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:50.913805  8058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:50.916044 32224 server_base.cc:1061] running on GCE node
W20260709 13:15:50.915731  8060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:50.920125 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:50.920419 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:50.920639 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602950920614 us; error 0 us; skew 500 ppm
I20260709 13:15:50.921240 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:50.925832 32224 webserver.cc:533] Webserver started at http://127.31.120.1:36985/ using document root <none> and password file <none>
I20260709 13:15:50.926412 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:50.926643 32224 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:15:50.928603  7468 consensus_peers.cc:597] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.931558 32224 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
W20260709 13:15:50.931648  7394 consensus_peers.cc:597] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.939396  8066 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:50.940275 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:15:50.940707 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "2b2b311085b64300925884dec9a28c7d"
format_stamp: "Formatted at 2026-07-09 13:15:37 on dist-test-slave-pm98"
I20260709 13:15:50.941207 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:15:50.959797  7320 consensus_peers.cc:597] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:50.974644 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:50.976230 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:50.978128 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:50.985666  8074 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:15:50.999097  7468 consensus_peers.cc:597] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:51.009857  7320 consensus_peers.cc:597] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:51.010419  7394 consensus_peers.cc:597] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:51.031451  7394 consensus_peers.cc:597] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:51.048720  7468 consensus_peers.cc:597] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.274698 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:15:51.275049 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.292s	user 0.010s	sys 0.000s
I20260709 13:15:51.275292 32224 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:15:51.279832  8074 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.292678  8074 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.293628  8074 tablet_bootstrap.cc:492] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.294248  8074 ts_tablet_manager.cc:1403] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 13:15:51.296818  8074 raft_consensus.cc:359] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.297508  8074 raft_consensus.cc:740] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.298195  8074 consensus_queue.cc:260] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.300850  8074 ts_tablet_manager.cc:1434] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:15:51.301648  8074 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.320225  8074 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.321111  8074 tablet_bootstrap.cc:492] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.321743  8074 ts_tablet_manager.cc:1403] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.012s
I20260709 13:15:51.323778  8074 raft_consensus.cc:359] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.324318  8074 raft_consensus.cc:740] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.324918  8074 consensus_queue.cc:260] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.326169  8074 ts_tablet_manager.cc:1434] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:51.326792  8074 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.364090  8074 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.365005  8074 tablet_bootstrap.cc:492] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.365608  8074 ts_tablet_manager.cc:1403] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.016s
I20260709 13:15:51.368991  8074 raft_consensus.cc:359] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.369628  8074 raft_consensus.cc:740] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.370288  8074 consensus_queue.cc:260] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.372133  8074 ts_tablet_manager.cc:1434] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:15:51.373605  8074 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.390074  8074 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.391072  8074 tablet_bootstrap.cc:492] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.391976  8074 ts_tablet_manager.cc:1403] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:15:51.395944  8074 raft_consensus.cc:359] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.396737  8074 raft_consensus.cc:740] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.397523  8074 consensus_queue.cc:260] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.399646  8074 ts_tablet_manager.cc:1434] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:51.400525  8074 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.487021  8074 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.488071  8074 tablet_bootstrap.cc:492] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.488802  8074 ts_tablet_manager.cc:1403] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.089s	user 0.038s	sys 0.040s
I20260709 13:15:51.491345  8074 raft_consensus.cc:359] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.491941  8074 raft_consensus.cc:740] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.492501  8074 consensus_queue.cc:260] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.494150  8074 ts_tablet_manager.cc:1434] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:15:51.494791  8074 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.506245  8074 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.506994  8074 tablet_bootstrap.cc:492] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.507514  8074 ts_tablet_manager.cc:1403] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:15:51.509414  8074 raft_consensus.cc:359] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.509912  8074 raft_consensus.cc:740] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.510450  8074 consensus_queue.cc:260] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.511873  8074 ts_tablet_manager.cc:1434] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:51.512477  8074 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.526409  8074 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.527292  8074 tablet_bootstrap.cc:492] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.528023  8074 ts_tablet_manager.cc:1403] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:15:51.530001  8074 raft_consensus.cc:359] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.530498  8074 raft_consensus.cc:740] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.531071  8074 consensus_queue.cc:260] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.532859  8074 ts_tablet_manager.cc:1434] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:51.533588  8074 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.547642 32224 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:15:51.548151 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.273s	user 0.239s	sys 0.006s
I20260709 13:15:51.567816  8074 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.568662  8074 tablet_bootstrap.cc:492] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.569216  8074 ts_tablet_manager.cc:1403] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.023s
I20260709 13:15:51.571342  8074 raft_consensus.cc:359] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.571993  8074 raft_consensus.cc:740] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.572834  8074 consensus_queue.cc:260] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.575078  8074 ts_tablet_manager.cc:1434] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:51.575918  8074 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.592595  8074 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.594053  8074 tablet_bootstrap.cc:492] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.594908  8074 ts_tablet_manager.cc:1403] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:15:51.598184  8074 raft_consensus.cc:359] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.599092  8074 raft_consensus.cc:740] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.600013  8074 consensus_queue.cc:260] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.602690  8074 ts_tablet_manager.cc:1434] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:15:51.603747  8074 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.626374  8074 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.627553  8074 tablet_bootstrap.cc:492] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.628196  8074 ts_tablet_manager.cc:1403] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.012s
I20260709 13:15:51.631130  8074 raft_consensus.cc:359] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.632556  8074 raft_consensus.cc:740] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.633459  8074 consensus_queue.cc:260] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.634861  7999 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.635442  7999 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.635788  8074 ts_tablet_manager.cc:1434] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:51.636722  8074 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.637591  7999 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.640285  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:51.641986  7468 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.642494  7471 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:51.643268  8079 raft_consensus.cc:2749] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:51.643746  8078 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.644286  8078 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.646315  8078 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:51.647857  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:51.654363  7320 leader_election.cc:336] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.654820  7320 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:51.655546  8078 raft_consensus.cc:2749] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:51.669026  8074 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.670141  8074 tablet_bootstrap.cc:492] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.670902  8074 ts_tablet_manager.cc:1403] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.005s
I20260709 13:15:51.674431  8074 raft_consensus.cc:359] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:51.675359  8074 raft_consensus.cc:740] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.676278  8074 consensus_queue.cc:260] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:51.679095  8074 ts_tablet_manager.cc:1434] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:15:51.680218  8074 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.685798  8075 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.686583  8075 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.689424  8075 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.693646  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:51.697531  7394 leader_election.cc:336] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.697999  7394 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:51.698849  8075 raft_consensus.cc:2749] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:51.703190  7999 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.703902  7999 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.708033  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:51.708732  8074 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.708931  7355 raft_consensus.cc:2468] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:51.709735  8074 tablet_bootstrap.cc:492] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.710446  8074 ts_tablet_manager.cc:1403] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.005s
I20260709 13:15:51.710902  7471 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:51.712182  8079 raft_consensus.cc:2804] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:51.712716  8079 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.712827  8074 raft_consensus.cc:359] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.713352  8079 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:51.713619  8074 raft_consensus.cc:740] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.714481  8074 consensus_queue.cc:260] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:51.715719  7468 leader_election.cc:336] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.719197  7999 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.720729  8074 ts_tablet_manager.cc:1434] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 13:15:51.721596  8074 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.753579  8079 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.759152  8076 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.759776  8076 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.761286  8079 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.762110  8076 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:51.762676  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:51.763697  7355 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:51.770522  7394 leader_election.cc:336] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.772133  8074 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:51.773419  8074 tablet_bootstrap.cc:492] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:51.773520  7355 raft_consensus.cc:2468] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:51.774391  8074 ts_tablet_manager.cc:1403] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.011s
W20260709 13:15:51.771377  7468 leader_election.cc:336] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:51.775413  7471 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:51.776233  7999 raft_consensus.cc:2804] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:51.777586  7999 raft_consensus.cc:697] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:51.777942  8074 raft_consensus.cc:359] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.778956  8074 raft_consensus.cc:740] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:51.778577  7999 consensus_queue.cc:237] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.780133  8074 consensus_queue.cc:260] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.783154  8074 ts_tablet_manager.cc:1434] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:15:51.784103  8074 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:51.950860  8075 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.951283  8074 tablet_bootstrap.cc:700] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Time spent opening tablet: real 0.161s	user 0.047s	sys 0.104s
I20260709 13:15:51.951642  8075 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:51.955524  8075 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:51.957329  8075 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.957880  8075 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:51.960033  8077 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.960780  8075 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.962858  8078 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.963183  8138 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.963537  8078 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.966089  8078 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:51.967217  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:51.969030  8078 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.967792  7999 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.973721  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:51.974800  7504 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:51.975948  7987 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.975687  7999 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.976541  7987 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.978355  7999 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:51.978820  7987 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:51.979445  7987 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.991500  8078 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:51.996306  7850 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.997043  7850 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.008724  8140 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.009048  7850 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:52.017391  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.000439  8076 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.006531  8080 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.995177  7987 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.027412  8073 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.009284  8140 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.022048  8076 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.026196  8080 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.042013  7999 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:51.963769  8138 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.042580  8076 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.042646  7999 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.043881  8080 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.044268  8138 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.045246  7999 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.028739  8073 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.046376  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.046516  7504 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:52.047148  7355 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:51.960719  8077 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.049659  8077 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.029431  8078 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.051366  8074 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.052438  8074 tablet_bootstrap.cc:492] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:52.052707  7987 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.053236  8074 ts_tablet_manager.cc:1403] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.269s	user 0.066s	sys 0.107s
I20260709 13:15:52.063701  7850 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.064311  7850 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.066468  8140 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.068128  8074 raft_consensus.cc:359] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.068843  8074 raft_consensus.cc:740] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:52.068921  8140 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.069523  8140 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.069967  8074 consensus_queue.cc:260] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.074923  8074 ts_tablet_manager.cc:1434] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.001s
I20260709 13:15:52.075320  8073 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.076637  8074 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:52.077510  8144 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.078126  7850 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.055069  7178 catalog_manager.cc:5697] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:52.084362  7468 leader_election.cc:336] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.085098  7468 leader_election.cc:336] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.085963  7468 leader_election.cc:336] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.093832  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.094583  7355 raft_consensus.cc:2468] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.078100  8144 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.096989  8140 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.098064  8144 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.100910  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.101622  7355 raft_consensus.cc:2468] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.103477  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:52.092725  7468 leader_election.cc:336] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.104157  7355 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.112020  8147 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.112712  8147 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.114161  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.114938  7504 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:52.115428  8147 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.118896  8150 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.119452  8150 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.122045  8150 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.122571  8150 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.123240  8150 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.127203  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.127671  7471 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.128968  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.129290  8144 raft_consensus.cc:2804] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.129650  7357 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.129799  8144 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.130239  8144 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.130676  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.131688  7429 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.133252  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.133558  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.134073  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.134644  7429 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.134788  7357 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.135372  7356 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.137342  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:52.137974  7355 raft_consensus.cc:2468] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:52.139443  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.141253  7431 raft_consensus.cc:2468] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.139757  7471 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.150655  8148 raft_consensus.cc:2804] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.151103  8148 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.151513  8148 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.152663  7394 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.157011  8150 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.158313  8149 raft_consensus.cc:2749] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.159858  7394 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:52.161633  8153 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.161531  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.171799  8153 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.172307  8153 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.162945  8145 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
W20260709 13:15:52.172693  7468 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.172899  8145 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
W20260709 13:15:52.174175  7468 leader_election.cc:336] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.174902  7468 leader_election.cc:336] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.175539  8145 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:52.176342  8145 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.176846  8145 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.179142  8145 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:52.179157  8074 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.164677  8148 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.165992  7471 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.166311  7397 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.182000  8140 raft_consensus.cc:2804] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.182406  8140 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.182813  8140 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.182870  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:52.168294  7394 leader_election.cc:336] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.185310  7397 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.187304  8073 raft_consensus.cc:2804] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.187736  8073 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.188072  8073 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.189375  7471 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:52.190698  8080 raft_consensus.cc:2749] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.191962  7471 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.193758  8080 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.194182  8080 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.194563  8080 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.169277  8144 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.172123  7429 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:52.162468  8149 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.180614  8074 tablet_bootstrap.cc:492] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:52.204241  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.206504  8075 raft_consensus.cc:2804] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.211215  7504 raft_consensus.cc:2393] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.207183  7320 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.216135  8075 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.216473  8145 raft_consensus.cc:2804] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.216693  8075 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.217010  8145 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.217554  8145 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.219430  7320 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.210599  8080 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.220521  8146 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.220896  8146 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.221216  8146 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.227042  8075 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.213814  8074 ts_tablet_manager.cc:1403] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.138s	user 0.026s	sys 0.000s
I20260709 13:15:52.206290  8149 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.232210  7428 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:15:52.233132  8149 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.206967  7471 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.234273  7471 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.213817  8148 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
W20260709 13:15:52.204149  7394 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.230966  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.237603  7356 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:52.237890  7394 leader_election.cc:336] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.237780  8153 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:52.238705  7394 leader_election.cc:336] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.239895  8153 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.203593  8140 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.240082  8146 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.241971  8140 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.242117  8146 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.243280  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.244087  7504 raft_consensus.cc:2393] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.244084  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.244926  7355 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.234556  8080 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.234807  8147 raft_consensus.cc:2804] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.246227  8147 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.246632  8147 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.248144  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.206327  8144 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.235497  8150 raft_consensus.cc:2804] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.252028  8150 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.252426  8150 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.254524  7356 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:15:52.264534  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.265154  7353 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.265712  8075 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.258600  8074 raft_consensus.cc:359] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.261772  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.267499  7357 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:52.267532  7394 leader_election.cc:336] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.268236  7394 leader_election.cc:336] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.268497  7354 raft_consensus.cc:1275] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:52.268639  7394 leader_election.cc:336] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.257913  8145 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.263172  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:15:52.264124  7468 leader_election.cc:336] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.274657  7353 raft_consensus.cc:2468] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.275328  7352 raft_consensus.cc:2393] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in current term 2: Already voted for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in this term.
W20260709 13:15:52.275527  7468 leader_election.cc:336] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.276283  7468 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.276490  7357 raft_consensus.cc:2468] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.259479  8150 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.278367  7504 raft_consensus.cc:2393] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.279055  8150 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.278774  8080 consensus_queue.cc:1048] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:52.280351  7471 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
W20260709 13:15:52.268756  7320 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.212661  8073 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.282992  8073 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.281255  8150 raft_consensus.cc:2804] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.283743  8150 raft_consensus.cc:697] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.284523  8150 consensus_queue.cc:237] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.286212  7471 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.287097  8144 raft_consensus.cc:2804] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.287633  8144 raft_consensus.cc:697] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.288561  8144 consensus_queue.cc:237] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.289637  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.290211  7357 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.293521  8145 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.297104  7397 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.297941  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.298682  7356 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.298764  7468 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.299948  8073 raft_consensus.cc:2804] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.300490  8073 raft_consensus.cc:697] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:52.301025  7355 raft_consensus.cc:2468] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.301326  8073 consensus_queue.cc:237] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.303315  8140 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.303764  8140 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
W20260709 13:15:52.303869  7394 leader_election.cc:336] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.304289  8140 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.262199  8147 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.306233  8147 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.309005  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:15:52.309641  7353 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.312023  8074 raft_consensus.cc:740] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:52.312218  7468 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.313795  8147 raft_consensus.cc:2804] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.314213  8147 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.314596  8147 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.314895  8074 consensus_queue.cc:260] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.318612  7353 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.318894  7471 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.308665  7356 raft_consensus.cc:2468] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:15:52.323083  8074 ts_tablet_manager.cc:1434] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:15:52.322813  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:52.323764  7428 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:52.324070  7397 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.326484  8073 raft_consensus.cc:2804] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.326972  8073 raft_consensus.cc:697] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:52.328069  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.328343  8147 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.328734  7504 raft_consensus.cc:2468] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:52.329979  7357 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.329819  8073 consensus_queue.cc:237] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.330508  7468 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d
I20260709 13:15:52.333447  8147 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.333469  8140 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.335767  8140 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.335805  8148 raft_consensus.cc:2804] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.336418  8148 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.336802  8148 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.342207  8080 raft_consensus.cc:2804] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.342818  8080 raft_consensus.cc:697] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.343719  8080 consensus_queue.cc:237] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.345005  7178 catalog_manager.cc:5697] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.346279  7471 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.347065  8147 raft_consensus.cc:2804] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.347543  8147 raft_consensus.cc:697] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.348380  8147 consensus_queue.cc:237] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.348371  7180 catalog_manager.cc:5697] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.349432  7471 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.350473  8074 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:52.351388  8079 raft_consensus.cc:2804] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.351895  8079 raft_consensus.cc:697] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.352700  8079 consensus_queue.cc:237] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.356109  7320 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.357753  8145 raft_consensus.cc:2764] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:52.364037  7320 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.366580  7468 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:52.371407  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:52.372543  7504 raft_consensus.cc:2393] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.383270  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:52.384382  8138 raft_consensus.cc:2804] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.385077  8138 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.385600  8138 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.387161  7504 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:52.397059  7320 leader_election.cc:336] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.400089  7320 leader_election.cc:336] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.401201  7320 leader_election.cc:336] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.401731  7320 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.414175  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:52.414917  7320 leader_election.cc:336] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.415647  7320 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.417488  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:52.352835  8148 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.418300  7503 raft_consensus.cc:2393] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.389978  7987 raft_consensus.cc:2804] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.413038  8146 raft_consensus.cc:2749] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.422166  7987 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.423560  7320 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.433771  7987 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.434607  8145 raft_consensus.cc:2749] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.439157  8148 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.424121  7504 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:52.427690  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:52.431955  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:52.449103  7429 raft_consensus.cc:2393] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in current term 2: Already voted for candidate 5270a004873440ec9a5e8106b63a6a85 in this term.
I20260709 13:15:52.450501  7501 raft_consensus.cc:2393] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.440845  8138 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.426753  7394 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.457233  8073 raft_consensus.cc:2749] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.423496  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:52.458200  7428 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.466693  7428 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
W20260709 13:15:52.447085  7468 leader_election.cc:336] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.467980  7505 raft_consensus.cc:2393] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5270a004873440ec9a5e8106b63a6a85 in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:52.467064  7177 catalog_manager.cc:5697] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.476146  7177 catalog_manager.cc:5697] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.477617  7177 catalog_manager.cc:5697] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.469574  8138 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.479516  7987 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
W20260709 13:15:52.481297  7468 leader_election.cc:336] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.481597  7987 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:15:52.439234  7320 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
W20260709 13:15:52.482863  7468 leader_election.cc:336] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.486882  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:52.487562  7427 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.483860  7180 catalog_manager.cc:5697] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.489055  7394 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.491501  8145 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.491740  8073 raft_consensus.cc:2749] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:52.492465  7320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 53314.8us
I20260709 13:15:52.495074  7320 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.495797  8145 raft_consensus.cc:2804] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:52.496163  8145 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:15:52.496517  8145 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.502240  8183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:33107 every 8 connection(s)
I20260709 13:15:52.502889 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:33107
I20260709 13:15:52.505502  7427 raft_consensus.cc:2468] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.506619  7177 catalog_manager.cc:5697] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.510188  8145 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.517025  8074 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.523170  8074 tablet_bootstrap.cc:492] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:52.518308  7320 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.523979  8074 ts_tablet_manager.cc:1403] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.174s	user 0.017s	sys 0.010s
I20260709 13:15:52.524221  8146 raft_consensus.cc:2804] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.525043  8146 raft_consensus.cc:697] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
W20260709 13:15:52.525509  7320 leader_election.cc:336] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.526240  8146 consensus_queue.cc:237] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.529276  8074 raft_consensus.cc:359] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.530043  8074 raft_consensus.cc:740] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:52.531118  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:52.531901  7427 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.535212  7394 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.537757  8074 consensus_queue.cc:260] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.562438  8074 ts_tablet_manager.cc:1434] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.001s
I20260709 13:15:52.563391  8074 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:52.538105  8145 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Requested vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
W20260709 13:15:52.540679  7320 leader_election.cc:336] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.564913  7320 leader_election.cc:336] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.565322  7320 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:15:52.565595  7427 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.566112  7987 raft_consensus.cc:2749] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.543427  7468 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:52.566795  8073 raft_consensus.cc:2749] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:52.567495  7320 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.568334  7468 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:52.568796  8138 raft_consensus.cc:2804] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.576047  8138 raft_consensus.cc:697] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.576570  7468 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:52.576949  8138 consensus_queue.cc:237] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:15:52.577435  7468 leader_election.cc:336] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.577841  7468 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:15:52.578701  7468 leader_election.cc:336] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.579387  8077 raft_consensus.cc:2804] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.580624  8079 raft_consensus.cc:2804] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.581130  8079 raft_consensus.cc:697] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
W20260709 13:15:52.579350  7468 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.586110  7468 leader_election.cc:336] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.587069  7468 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
W20260709 13:15:52.587756  7468 leader_election.cc:336] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:15:52.588183  7468 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:52.594834  8148 raft_consensus.cc:2749] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.582000  8079 consensus_queue.cc:237] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.596426  8077 raft_consensus.cc:697] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Leader. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:15:52.597359  8077 consensus_queue.cc:237] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.599970  8081 raft_consensus.cc:2749] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:52.613622  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:15:52.620844  7177 catalog_manager.cc:5697] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.614491  7427 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.647172  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:52.679451  7501 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.687778  7180 catalog_manager.cc:5697] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.706229  8184 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41611
I20260709 13:15:52.706684  8184 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:52.707597  8184 heartbeater.cc:507] Master 127.31.120.62:41611 requested a full tablet report, sending...
I20260709 13:15:52.710240  7177 catalog_manager.cc:5697] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.715023  7427 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:52.715210  7177 catalog_manager.cc:5697] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4). New cstate: current_term: 2 leader_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.716387  7320 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.716436  8124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:52.717643  8145 raft_consensus.cc:2804] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.720156  8145 raft_consensus.cc:697] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:52.721030  8145 consensus_queue.cc:237] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:52.716725  8125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:52.730937  7352 raft_consensus.cc:1275] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.726279  7501 raft_consensus.cc:2468] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:52.733472  7320 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:52.735427  8146 raft_consensus.cc:2804] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:52.736140  8146 raft_consensus.cc:697] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:52.736987  8146 consensus_queue.cc:237] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:52.738891  7320 leader_election.cc:343] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:52.739708  8148 consensus_queue.cc:1048] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:15:52.741276  7320 leader_election.cc:343] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:52.759649  7177 ts_manager.cc:194] Re-registered known tserver with Master: 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:52.767938  8074 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.772070  8074 tablet_bootstrap.cc:492] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:52.772763  8074 ts_tablet_manager.cc:1403] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.210s	user 0.015s	sys 0.010s
I20260709 13:15:52.777763  7180 catalog_manager.cc:5697] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:52.792366  7357 raft_consensus.cc:1275] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.793320  8120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:52.794637  8074 raft_consensus.cc:359] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.795526  8074 raft_consensus.cc:740] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:52.796520  8074 consensus_queue.cc:260] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.785926  8083 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.794793  8148 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:15:52.812923  8083 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:52.816256  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:52.816993  8117 raft_consensus.cc:3060] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.818126  8083 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:52.820384  8083 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.821678  8120 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:52.827697  7352 raft_consensus.cc:1275] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.829594  8083 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:52.841692  8083 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:15:52.844488  7357 raft_consensus.cc:1275] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.861569  8073 consensus_queue.cc:1048] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:15:52.866601  7352 raft_consensus.cc:1275] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.868433  8079 consensus_queue.cc:1048] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:52.861749  8147 consensus_queue.cc:1048] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:15:52.878999  8117 raft_consensus.cc:2468] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.900677  8074 ts_tablet_manager.cc:1434] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.128s	user 0.004s	sys 0.011s
I20260709 13:15:52.901574  8074 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:52.914776  8120 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:15:52.922910  8206 raft_consensus.cc:493] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.923503  8206 raft_consensus.cc:515] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:15:52.925683  8206 leader_election.cc:290] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:52.940610  7355 raft_consensus.cc:1275] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:52.999414  7468 consensus_peers.cc:597] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:52.994824  8206 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.001608  8206 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.003280  7351 raft_consensus.cc:1275] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:52.960032  8075 consensus_queue.cc:1048] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:15:53.007534  8206 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:15:53.008527  8120 raft_consensus.cc:3060] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.019002  8118 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.030803  7179 catalog_manager.cc:5697] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 2, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:53.039376  8077 consensus_queue.cc:1048] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.040328  8118 raft_consensus.cc:1275] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.044891  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:53.046298  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:53.047866  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:53.048940  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:15:53.059288  7468 consensus_peers.cc:597] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:53.061033  7468 consensus_peers.cc:597] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:53.062382  7468 consensus_peers.cc:597] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.069545  8148 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.070636  7430 raft_consensus.cc:1275] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.077148  8120 raft_consensus.cc:1275] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.082849  8142 consensus_queue.cc:1048] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.101651  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:53.152050  8074 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.124413  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:15:53.172345  7397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 68622.3us
I20260709 13:15:53.163226  7177 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53382
I20260709 13:15:53.181114  8074 tablet_bootstrap.cc:492] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.182209  8074 ts_tablet_manager.cc:1403] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.281s	user 0.023s	sys 0.020s
I20260709 13:15:53.178448  7429 raft_consensus.cc:2393] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b2b311085b64300925884dec9a28c7d in current term 2: Already voted for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in this term.
I20260709 13:15:53.189184  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:15:53.148526  8067 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:53.198815  8118 raft_consensus.cc:1275] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.211544  8206 raft_consensus.cc:2749] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:53.214278  8119 raft_consensus.cc:1275] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:53.217744  8067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 90625.8us
I20260709 13:15:53.279544  7431 raft_consensus.cc:1275] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.279809  8067 leader_election.cc:304] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:53.294883  8184 heartbeater.cc:499] Master 127.31.120.62:41611 was elected leader, sending a full tablet report...
W20260709 13:15:53.297065  8218 log.cc:927] Time spent T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:15:53.299181  8207 raft_consensus.cc:2749] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:53.316614  8074 raft_consensus.cc:359] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
W20260709 13:15:53.322458  7394 consensus_peers.cc:597] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.317790  8074 raft_consensus.cc:740] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.326872  8074 consensus_queue.cc:260] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.331436  8074 ts_tablet_manager.cc:1434] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.149s	user 0.009s	sys 0.001s
I20260709 13:15:53.324379  7424 raft_consensus.cc:1275] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.337877  8206 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.336535  8067 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:15:53.368203  7503 raft_consensus.cc:1275] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.359735  8206 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.377588  8206 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
W20260709 13:15:53.378464  8067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 48006us
I20260709 13:15:53.379261  8207 raft_consensus.cc:2749] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:53.384102  8123 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.385526  8074 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.393968  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:53.398186  7425 raft_consensus.cc:1275] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 49f7f0d3dec548bbb5538d2b1ddb4415: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:53.417369  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 76179us
I20260709 13:15:53.366253  8231 consensus_queue.cc:1048] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:15:53.384928  8230 consensus_queue.cc:1048] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:15:53.427258  8225 consensus_queue.cc:1048] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:15:53.434213  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:15:53.428064  7320 consensus_peers.cc:597] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.439642  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:15:53.441196  8147 consensus_queue.cc:1048] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:53.444437  7468 consensus_peers.cc:597] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.445077  7468 consensus_peers.cc:597] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.446487  8125 raft_consensus.cc:3060] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.458314  7505 raft_consensus.cc:1275] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:53.438261  7320 consensus_peers.cc:597] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:53.466991  8070 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:53.473423  7850 consensus_queue.cc:1048] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.481999  8123 raft_consensus.cc:1275] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.482678  8074 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.489204  8070 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:15:53.490096  8206 raft_consensus.cc:2749] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:53.499125  7394 consensus_peers.cc:597] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:53.500241  8226 consensus_queue.cc:1048] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.505573  8074 tablet_bootstrap.cc:492] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.506559  8074 ts_tablet_manager.cc:1403] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.012s
I20260709 13:15:53.524116  7850 consensus_queue.cc:1048] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.533229  8196 consensus_queue.cc:1048] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.530243  8231 consensus_queue.cc:1048] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:15:53.535193  8074 raft_consensus.cc:359] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.535961  8074 raft_consensus.cc:740] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.534644  8125 raft_consensus.cc:1275] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.547776  8145 consensus_queue.cc:1048] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.548295  8074 consensus_queue.cc:260] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.550974  8074 ts_tablet_manager.cc:1434] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:15:53.552297  8074 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.552590  8206 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:53.596385  7320 consensus_peers.cc:597] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.598778  7320 consensus_peers.cc:597] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.600768  7320 consensus_peers.cc:597] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.603978  7468 consensus_peers.cc:597] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.605904  7468 consensus_peers.cc:597] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.606750  7468 consensus_peers.cc:597] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.607282  7468 consensus_peers.cc:597] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.620774  7394 consensus_peers.cc:597] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.623075  7468 consensus_peers.cc:597] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.623842  7468 consensus_peers.cc:597] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:53.636988  8074 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.638309  8074 tablet_bootstrap.cc:492] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.639086  8074 ts_tablet_manager.cc:1403] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.012s
I20260709 13:15:53.641724  8074 raft_consensus.cc:359] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.642619  8074 raft_consensus.cc:740] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.643493  8074 consensus_queue.cc:260] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.646019  8074 ts_tablet_manager.cc:1434] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:53.647156  8074 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
W20260709 13:15:53.690470  7468 consensus_peers.cc:597] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:53.697028  7468 consensus_peers.cc:597] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:53.707296  8074 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.708143  8074 tablet_bootstrap.cc:492] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.708811  8074 ts_tablet_manager.cc:1403] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.002s
I20260709 13:15:53.711014  8074 raft_consensus.cc:359] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.712744  8074 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.713450  8074 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.715544  8074 ts_tablet_manager.cc:1434] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:53.716395  8074 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.731292  8074 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.732169  8074 tablet_bootstrap.cc:492] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.732744  8074 ts_tablet_manager.cc:1403] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:15:53.735968  8074 raft_consensus.cc:359] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.736729  8074 raft_consensus.cc:740] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.737392  8074 consensus_queue.cc:260] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.739485  8074 ts_tablet_manager.cc:1434] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:53.740303  8074 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.755105  8074 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.756127  8074 tablet_bootstrap.cc:492] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.756807  8074 ts_tablet_manager.cc:1403] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:15:53.759351  8074 raft_consensus.cc:359] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.760058  8074 raft_consensus.cc:740] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.760721  8074 consensus_queue.cc:260] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.763502  8074 ts_tablet_manager.cc:1434] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:15:53.764636  8074 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.779486  8074 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.780507  8074 tablet_bootstrap.cc:492] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.781167  8074 ts_tablet_manager.cc:1403] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:15:53.783644  8074 raft_consensus.cc:359] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.784394  8074 raft_consensus.cc:740] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.785006  8074 consensus_queue.cc:260] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.786788  8074 ts_tablet_manager.cc:1434] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:15:53.788307  8074 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.801606  8074 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.802476  8074 tablet_bootstrap.cc:492] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.803102  8074 ts_tablet_manager.cc:1403] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:15:53.805025  8074 raft_consensus.cc:359] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.805589  8074 raft_consensus.cc:740] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.806188  8074 consensus_queue.cc:260] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.808218  8074 ts_tablet_manager.cc:1434] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:15:53.809293  8074 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.822010  8074 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.822899  8074 tablet_bootstrap.cc:492] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.823815  8074 ts_tablet_manager.cc:1403] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:15:53.825999  8074 raft_consensus.cc:359] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.826454  8074 raft_consensus.cc:740] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.827073  8074 consensus_queue.cc:260] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.828995  8074 ts_tablet_manager.cc:1434] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:53.830369  8074 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.845069  8074 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.846015  8074 tablet_bootstrap.cc:492] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.846668  8074 ts_tablet_manager.cc:1403] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:15:53.848945  8074 raft_consensus.cc:359] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.849606  8074 raft_consensus.cc:740] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.850247  8074 consensus_queue.cc:260] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.852051  8074 ts_tablet_manager.cc:1434] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:15:53.852837  8074 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.865324  8074 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.866169  8074 tablet_bootstrap.cc:492] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.866709  8074 ts_tablet_manager.cc:1403] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:15:53.868705  8074 raft_consensus.cc:359] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.869199  8074 raft_consensus.cc:740] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.869834  8074 consensus_queue.cc:260] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:53.871433  8074 ts_tablet_manager.cc:1434] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:53.872478  8074 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.888185  8074 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.889515  8074 tablet_bootstrap.cc:492] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.890162  8074 ts_tablet_manager.cc:1403] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.001s
I20260709 13:15:53.896489  8074 raft_consensus.cc:359] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.897161  8074 raft_consensus.cc:740] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.897742  8074 consensus_queue.cc:260] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:53.899808  8074 ts_tablet_manager.cc:1434] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:53.900569  8074 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.945658  8121 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.958987  8120 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.962154  8074 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.963160  8074 tablet_bootstrap.cc:492] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.963989  8074 ts_tablet_manager.cc:1403] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.064s	user 0.031s	sys 0.031s
I20260709 13:15:53.966462  8074 raft_consensus.cc:359] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.967108  8074 raft_consensus.cc:740] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:53.967911  8074 consensus_queue.cc:260] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:53.970009  8074 ts_tablet_manager.cc:1434] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:15:53.970767  8074 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:53.991043  8074 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:53.992044  8074 tablet_bootstrap.cc:492] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:53.992729  8074 ts_tablet_manager.cc:1403] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:15:54.008329  8074 raft_consensus.cc:359] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.009473  8074 raft_consensus.cc:740] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.010543  8074 consensus_queue.cc:260] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.020584  8074 ts_tablet_manager.cc:1434] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:15:54.021481  8074 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.049057  8074 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.050179  8074 tablet_bootstrap.cc:492] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.050923  8074 ts_tablet_manager.cc:1403] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.003s
I20260709 13:15:54.053890  8074 raft_consensus.cc:359] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.054616  8074 raft_consensus.cc:740] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.055424  8074 consensus_queue.cc:260] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.057749  8074 ts_tablet_manager.cc:1434] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:54.058867  8074 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.102963  8074 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.104038  8074 tablet_bootstrap.cc:492] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.104756  8074 ts_tablet_manager.cc:1403] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.012s
I20260709 13:15:54.107489  8074 raft_consensus.cc:359] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.108294  8074 raft_consensus.cc:740] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.108980  8074 consensus_queue.cc:260] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.111078  8074 ts_tablet_manager.cc:1434] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:15:54.112236  8074 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.127918  8074 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.128798  8074 tablet_bootstrap.cc:492] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.129444  8074 ts_tablet_manager.cc:1403] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:15:54.131546  8074 raft_consensus.cc:359] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.132077  8074 raft_consensus.cc:740] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.132670  8074 consensus_queue.cc:260] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.134563  8074 ts_tablet_manager.cc:1434] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:54.135671  8074 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.153201  8074 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.154031  8074 tablet_bootstrap.cc:492] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.154699  8074 ts_tablet_manager.cc:1403] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:15:54.156510  8074 raft_consensus.cc:359] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.157006  8074 raft_consensus.cc:740] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.157586  8074 consensus_queue.cc:260] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.159045  8074 ts_tablet_manager.cc:1434] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:54.160706  8074 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.174288  8074 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.175194  8074 tablet_bootstrap.cc:492] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.176190  8074 ts_tablet_manager.cc:1403] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 13:15:54.178464  8074 raft_consensus.cc:359] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.179144  8074 raft_consensus.cc:740] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.179850  8074 consensus_queue.cc:260] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.181722  8074 ts_tablet_manager.cc:1434] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 13:15:54.182417  8074 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.196205  8074 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.197124  8074 tablet_bootstrap.cc:492] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.197778  8074 ts_tablet_manager.cc:1403] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:15:54.199750  8074 raft_consensus.cc:359] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.200379  8074 raft_consensus.cc:740] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.200994  8074 consensus_queue.cc:260] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.202402  8074 ts_tablet_manager.cc:1434] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:54.203064  8074 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.213224  8153 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.213907  8153 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.217715  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:54.218380  7505 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:15:54.219516  8120 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:54.219674  7394 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:54.220569  8226 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:54.220952  8226 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.221247  8226 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.228276  8226 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.231925  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:54.232070  8123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:54.232558  7505 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.233043  8123 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:15:54.233549  8121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:54.234234  8121 raft_consensus.cc:3060] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.219559  8153 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:54.240201  7505 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 3.
I20260709 13:15:54.243244  8226 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:54.244789  7394 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:15:54.245477  8121 raft_consensus.cc:2468] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 3.
I20260709 13:15:54.247191  8226 raft_consensus.cc:2804] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:54.247840  8226 raft_consensus.cc:697] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 3 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:15:54.248842  8226 consensus_queue.cc:237] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.252516  8074 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.253506  8074 tablet_bootstrap.cc:492] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.254112  8074 ts_tablet_manager.cc:1403] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.051s	user 0.025s	sys 0.011s
I20260709 13:15:54.258744  8074 raft_consensus.cc:359] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.259408  8074 raft_consensus.cc:740] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.260685  8074 consensus_queue.cc:260] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.263188  8074 ts_tablet_manager.cc:1434] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:54.263996  8074 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.276584  7177 catalog_manager.cc:5697] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 3, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:54.286487  8074 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.287842  8074 tablet_bootstrap.cc:492] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.288713  8074 ts_tablet_manager.cc:1403] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:15:54.291819  8074 raft_consensus.cc:359] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.292727  8074 raft_consensus.cc:740] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.294598  8074 consensus_queue.cc:260] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.297760  8074 ts_tablet_manager.cc:1434] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:54.301200  8074 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.343211  8074 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.344164  8074 tablet_bootstrap.cc:492] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.344789  8074 ts_tablet_manager.cc:1403] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.044s	user 0.029s	sys 0.011s
I20260709 13:15:54.348875  8074 raft_consensus.cc:359] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.349481  8074 raft_consensus.cc:740] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.350087  8074 consensus_queue.cc:260] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:15:54.351704  8074 ts_tablet_manager.cc:1434] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:54.352564  8074 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.413836  8074 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.415953  8074 tablet_bootstrap.cc:492] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.417477  8074 ts_tablet_manager.cc:1403] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.004s
I20260709 13:15:54.430390  8074 raft_consensus.cc:359] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.432082  8074 raft_consensus.cc:740] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.435849  8074 consensus_queue.cc:260] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.453382  8074 ts_tablet_manager.cc:1434] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.007s
I20260709 13:15:54.455040  8074 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.464489  7850 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.465516  7850 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.472918  7850 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:54.474871  8123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d" is_pre_election: true
I20260709 13:15:54.496928  8123 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 1.
I20260709 13:15:54.488013  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:15:54.499873  7505 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 2.
I20260709 13:15:54.499898  7320 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:54.504730  8123 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:54.512025  8196 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:54.512509  8196 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.512864  8196 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.513566  8074 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.514637  8074 tablet_bootstrap.cc:492] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.515753  8074 ts_tablet_manager.cc:1403] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.061s	user 0.022s	sys 0.008s
I20260709 13:15:54.537045  8074 raft_consensus.cc:359] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.544754  8074 raft_consensus.cc:740] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.548597  8074 consensus_queue.cc:260] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:15:54.550802  8074 ts_tablet_manager.cc:1434] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:15:54.558805  8074 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.576536  8120 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:54.636904  7320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 105307us
I20260709 13:15:54.649700  8196 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.686492  8074 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.693766  8074 tablet_bootstrap.cc:492] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.695369  8074 ts_tablet_manager.cc:1403] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.137s	user 0.024s	sys 0.001s
W20260709 13:15:54.716542  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 263913us
I20260709 13:15:54.742163  8074 raft_consensus.cc:359] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.748309  8074 raft_consensus.cc:740] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.741885  8196 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:15:54.754711  8074 consensus_queue.cc:260] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.774619  8074 ts_tablet_manager.cc:1434] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.003s
W20260709 13:15:54.781980  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 47092.4us
I20260709 13:15:54.793931  8124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2b2b311085b64300925884dec9a28c7d"
I20260709 13:15:54.794734  8124 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.796738  8074 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.805867  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:15:54.806787  7501 raft_consensus.cc:3060] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.846262  7501 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 3.
I20260709 13:15:54.853266  7320 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:15:54.854560  8295 raft_consensus.cc:2804] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:54.855404  8295 raft_consensus.cc:697] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 LEADER]: Becoming Leader. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:15:54.856611  8295 consensus_queue.cc:237] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.868800  8117 raft_consensus.cc:1275] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:54.870262  8123 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:54.871940  8124 raft_consensus.cc:2468] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71e04cb9c24b4ddd9d56199a77f5aa89 in term 3.
I20260709 13:15:54.896401  8074 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.899214  8074 tablet_bootstrap.cc:492] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.900017  8074 ts_tablet_manager.cc:1403] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.011s
I20260709 13:15:54.903169  8074 raft_consensus.cc:359] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.904973  8074 raft_consensus.cc:740] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.905918  8074 consensus_queue.cc:260] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:15:54.896826  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 57354.6us
W20260709 13:15:54.897061  7471 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 52391.1us
I20260709 13:15:54.908358  8074 ts_tablet_manager.cc:1434] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:15:54.909214  8074 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Bootstrap starting.
I20260709 13:15:54.916493  7501 raft_consensus.cc:1275] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:54.922955  8211 consensus_queue.cc:1048] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:54.946959  8074 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:54.947954  8074 tablet_bootstrap.cc:492] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Bootstrap complete.
I20260709 13:15:54.948632  8074 ts_tablet_manager.cc:1403] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.013s
I20260709 13:15:54.951285  8074 raft_consensus.cc:359] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.951980  8074 raft_consensus.cc:740] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.957312  8074 consensus_queue.cc:260] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:15:54.970438  8074 ts_tablet_manager.cc:1434] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.011s
I20260709 13:15:54.996912  7180 catalog_manager.cc:5697] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 reported cstate change: term changed from 1 to 3, leader changed from 2b2b311085b64300925884dec9a28c7d (127.31.120.1) to 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2). New cstate: current_term: 3 leader_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:55.046702  8289 consensus_queue.cc:1048] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:55.053907  8121 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:55.259375  8121 raft_consensus.cc:1275] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:55.261055  8308 consensus_queue.cc:1048] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:55.275497  7504 raft_consensus.cc:1275] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 71e04cb9c24b4ddd9d56199a77f5aa89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:55.277233  8295 consensus_queue.cc:1048] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:55.302543  8117 raft_consensus.cc:3060] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:02.573814 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:02.654608  7505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:02.655200  7505 raft_consensus.cc:606] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:02.657482  7505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:02.657984  7505 raft_consensus.cc:606] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:02.660496  7505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:02.661053  7505 raft_consensus.cc:606] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:02.674616  7352 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f"
dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:59012
I20260709 13:16:02.675113  7352 raft_consensus.cc:606] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:02.677623  7505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:02.678107  7505 raft_consensus.cc:606] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:02.681321  7505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:02.681843  7505 raft_consensus.cc:606] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:02.682958 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:02.684331 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:02.798507  8328 raft_consensus.cc:993] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.799018  8236 raft_consensus.cc:1081] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.800626  8121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:02.801452  8121 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:02.801892  8121 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.807886  8121 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.809459  8121 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:02.810604  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:02.811286  7505 raft_consensus.cc:3055] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:02.811565  7505 raft_consensus.cc:740] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:02.812381  7505 consensus_queue.cc:260] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.813476  7505 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.817607  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:02.819058  7426 raft_consensus.cc:3060] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.820868  7505 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:02.822579  8067 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:16:02.824496  8340 raft_consensus.cc:2804] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:02.826128  8340 raft_consensus.cc:697] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:02.826401  7426 raft_consensus.cc:2468] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:02.828400  8340 consensus_queue.cc:237] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.836948  8236 raft_consensus.cc:993] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.837397  8236 raft_consensus.cc:1081] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.839069  8121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:02.839514  8121 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:02.839833  8121 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:02.840700  7178 catalog_manager.cc:5697] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:02.845490  8121 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.849215  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:02.850338  7426 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:02.857139  7426 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:02.865971  8328 raft_consensus.cc:993] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.866565  8236 raft_consensus.cc:1081] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:02.868449  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:02.869035  8121 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:02.870105  8117 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:02.870491  8117 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.870622  8067 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:02.871934  8340 raft_consensus.cc:2804] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:02.872467  8340 raft_consensus.cc:697] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:02.872859  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:02.873741  7505 raft_consensus.cc:3055] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:02.873391  8340 consensus_queue.cc:237] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.874127  7505 raft_consensus.cc:740] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:02.875211  7505 consensus_queue.cc:260] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.880182  7505 raft_consensus.cc:3060] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:02.893695  8117 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.899638  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:02.901283  7426 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.904594  8117 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:02.910678  7426 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:02.912012  8067 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:02.913225  8340 raft_consensus.cc:2804] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:02.914436  8340 raft_consensus.cc:697] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:02.915284  8340 consensus_queue.cc:237] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.917217  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:02.918090  7504 raft_consensus.cc:3055] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:02.918467  7504 raft_consensus.cc:740] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:02.919941  7504 consensus_queue.cc:260] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:02.927680  7504 raft_consensus.cc:3060] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:02.900565  7505 raft_consensus.cc:2468] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:02.937395  7504 raft_consensus.cc:2468] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:02.941195  7178 catalog_manager.cc:5697] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:02.942566  7178 catalog_manager.cc:5697] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:03.033740  8349 raft_consensus.cc:993] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.034190  8328 raft_consensus.cc:1081] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.035571  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:03.036095  8117 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.036394  8117 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.041569  8117 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.044684  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.044855  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:03.045344  7504 raft_consensus.cc:3055] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:03.045478  7352 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.045692  8117 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:03.045732  7504 raft_consensus.cc:740] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:03.047147  7504 consensus_queue.cc:260] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.048671  8332 raft_consensus.cc:993] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.049052  7504 raft_consensus.cc:3060] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.049297  8336 raft_consensus.cc:1081] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.050976  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49082
I20260709 13:16:03.051486  8117 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.051859  8117 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.051916  7352 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:03.053283  8070 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:03.054055  8340 raft_consensus.cc:2804] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:03.054522  8340 raft_consensus.cc:697] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:03.055152  7504 raft_consensus.cc:2468] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:03.055437  8340 consensus_queue.cc:237] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.057524  8117 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.059180  8117 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:03.059960  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.060482  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:03.061086  7352 raft_consensus.cc:3055] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:03.061372  7352 raft_consensus.cc:740] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:03.062100  7352 consensus_queue.cc:260] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.060632  7504 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.063248  7352 raft_consensus.cc:3060] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.067957  7178 catalog_manager.cc:5697] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.070374  7504 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:03.070652  7352 raft_consensus.cc:2468] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:03.071566  8067 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 
I20260709 13:16:03.072719  8340 raft_consensus.cc:2804] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:03.073201  8340 raft_consensus.cc:697] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:03.074410  8340 consensus_queue.cc:237] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.085021  7178 catalog_manager.cc:5697] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.162914  8236 raft_consensus.cc:993] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:03.163344  8236 raft_consensus.cc:1081] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:03.165145  7427 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d"
dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:40220
I20260709 13:16:03.166730  7427 raft_consensus.cc:493] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.167117  7427 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.173272  7427 raft_consensus.cc:515] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.177330  7427 leader_election.cc:290] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:03.179231  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:03.179913  7352 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.187202  7352 raft_consensus.cc:2468] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:03.188201  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.188279  7397 leader_election.cc:304] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:03.188836  7504 raft_consensus.cc:3055] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:03.189141  7504 raft_consensus.cc:740] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:03.189185  8335 raft_consensus.cc:2804] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:03.189882  8335 raft_consensus.cc:697] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:03.190106  7504 consensus_queue.cc:260] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.190742  8335 consensus_queue.cc:237] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.191267  7504 raft_consensus.cc:3060] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.197223  7504 raft_consensus.cc:2468] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:03.200963  7178 catalog_manager.cc:5697] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.217270  7426 raft_consensus.cc:1275] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.218498  8340 consensus_queue.cc:1048] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.233906  7504 raft_consensus.cc:1275] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.235567  8341 consensus_queue.cc:1048] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:03.348707  7504 raft_consensus.cc:1275] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.349875  8341 consensus_queue.cc:1048] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.363468  7426 raft_consensus.cc:1275] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.365254  8359 consensus_queue.cc:1048] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.431941  7505 raft_consensus.cc:1275] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:03.433758  8359 consensus_queue.cc:1048] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.448051  7427 raft_consensus.cc:1275] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:03.449781  8340 consensus_queue.cc:1048] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.589684  7504 raft_consensus.cc:1275] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.591297  8340 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.600821  7353 raft_consensus.cc:1275] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.602533  8359 consensus_queue.cc:1048] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.615471  7352 raft_consensus.cc:1275] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.617561  8341 consensus_queue.cc:1048] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:03.631486  7504 raft_consensus.cc:1275] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:03.634507  8359 consensus_queue.cc:1048] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:03.664074  7353 raft_consensus.cc:1275] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:03.667966  8335 consensus_queue.cc:1048] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.686830 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:03.710870  7504 raft_consensus.cc:1275] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:03.713019  8335 consensus_queue.cc:1048] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:03.727255  7397 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 49276.8us
I20260709 13:16:03.794827 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:03.837618  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:03.838169  7504 raft_consensus.cc:606] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:03.841189  7353 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a"
dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
mode: GRACEFUL
new_leader_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:59012
I20260709 13:16:03.841717  7353 raft_consensus.cc:606] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Received request to transfer leadership to TS 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:03.845165  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:03.845815  7504 raft_consensus.cc:606] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:03.848435  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:03.849054  7504 raft_consensus.cc:606] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:03.851321  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:03.851814  7504 raft_consensus.cc:606] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:03.854177  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:03.854633  7504 raft_consensus.cc:606] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:03.855780 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:03.857220 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:03.869948  8328 raft_consensus.cc:993] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:03.870447  8236 raft_consensus.cc:1081] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:03.871742  7426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c"
dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:40220
I20260709 13:16:03.872200  7426 raft_consensus.cc:493] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.872495  7426 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.879016  7426 raft_consensus.cc:515] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.880856  7426 leader_election.cc:290] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:03.881623  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:03.882165  7353 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.882270  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.882807  7504 raft_consensus.cc:3055] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:03.883069  7504 raft_consensus.cc:740] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:03.883673  7504 consensus_queue.cc:260] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.884579  7504 raft_consensus.cc:3060] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.890427  7353 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:03.891637  7397 leader_election.cc:304] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:03.892350  8335 raft_consensus.cc:2804] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:03.892864  8335 raft_consensus.cc:697] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:03.893563  7504 raft_consensus.cc:2468] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:03.893687  8335 consensus_queue.cc:237] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:03.897598  8236 raft_consensus.cc:993] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.898108  8349 raft_consensus.cc:1081] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.899966  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:03.900561  8115 raft_consensus.cc:493] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.900939  8115 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.904767  7177 catalog_manager.cc:5697] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.907377  8115 raft_consensus.cc:515] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.909063  8115 leader_election.cc:290] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:03.909860  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.910036  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5aa9b772e6445c83184f5551976222" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:03.910491  7504 raft_consensus.cc:3055] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:03.910633  7353 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.910820  7504 raft_consensus.cc:740] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:03.911545  7504 consensus_queue.cc:260] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.912757  7504 raft_consensus.cc:3060] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.916970  7353 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:03.918376  8070 leader_election.cc:304] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:03.918717  7504 raft_consensus.cc:2468] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:03.919153  8367 raft_consensus.cc:2804] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:03.919721  8367 raft_consensus.cc:697] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:03.920516  8367 consensus_queue.cc:237] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:03.930336  8328 raft_consensus.cc:993] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.930817  8328 raft_consensus.cc:1081] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:03.932631  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:03.933209  8115 raft_consensus.cc:493] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.933704  8115 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.933068  7177 catalog_manager.cc:5697] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.943221  8115 raft_consensus.cc:515] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:03.944972  8115 leader_election.cc:290] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:03.945966  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:03.945878  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:03.946624  7427 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.946882  7504 raft_consensus.cc:3055] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:03.947252  7504 raft_consensus.cc:740] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:03.947978  7504 consensus_queue.cc:260] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:03.949076  7504 raft_consensus.cc:3060] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.953665  7427 raft_consensus.cc:2468] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:03.953845  7504 raft_consensus.cc:2468] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:03.954756  8067 leader_election.cc:304] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:03.955577  8340 raft_consensus.cc:2804] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:03.956092  8340 raft_consensus.cc:697] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:03.956805  8340 consensus_queue.cc:237] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:03.966042  7177 catalog_manager.cc:5697] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:04.195789  8349 raft_consensus.cc:993] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:04.196333  8328 raft_consensus.cc:1081] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:04.197772  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:04.198225  8115 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.198503  8115 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.202919  8115 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:04.204493  8115 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:04.205209  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:04.205528  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:04.205854  7427 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.206156  7504 raft_consensus.cc:3055] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:04.206526  7504 raft_consensus.cc:740] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:04.207074  7504 consensus_queue.cc:260] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:04.208050  7504 raft_consensus.cc:3060] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.210559  7427 raft_consensus.cc:2468] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:04.211707  8067 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:04.212443  8341 raft_consensus.cc:2804] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:04.212913  8341 raft_consensus.cc:697] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:04.213405  7504 raft_consensus.cc:2468] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:04.213771  8341 consensus_queue.cc:237] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:04.223696  7178 catalog_manager.cc:5697] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:04.233341  8349 raft_consensus.cc:993] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:04.233871  8328 raft_consensus.cc:1081] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:04.235944  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:04.236546  8115 raft_consensus.cc:493] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.236840  8115 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.241780  8115 raft_consensus.cc:515] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:04.244017  8115 leader_election.cc:290] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:04.244393  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:04.244552  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:04.245179  7504 raft_consensus.cc:3055] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:04.245215  7353 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.245723  7504 raft_consensus.cc:740] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:04.246742  7504 consensus_queue.cc:260] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:04.248160  7504 raft_consensus.cc:3060] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.257591  7353 raft_consensus.cc:2468] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:04.257658  7504 raft_consensus.cc:2468] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:04.258774  8070 leader_election.cc:304] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:04.259706  8341 raft_consensus.cc:2804] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:04.260231  8341 raft_consensus.cc:697] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:04.261363  8341 consensus_queue.cc:237] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:04.271917  7178 catalog_manager.cc:5697] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:04.277053  8332 raft_consensus.cc:993] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: : Instructing follower 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:04.277611  8332 raft_consensus.cc:1081] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Signalling peer 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:04.279747  7427 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a"
dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:40226
I20260709 13:16:04.280275  7427 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.280665  7427 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.286099  7427 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:04.288321  7427 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:04.289346  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:04.290275  7504 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.288825  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:04.292773  7353 raft_consensus.cc:3055] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:04.293048  7353 raft_consensus.cc:740] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:04.295387  7352 raft_consensus.cc:1275] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.293743  7353 consensus_queue.cc:260] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:04.296890  7353 raft_consensus.cc:3060] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:04.296950  8335 consensus_queue.cc:1048] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.299856  7504 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:04.301138  7394 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:04.301961  8335 raft_consensus.cc:2804] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:04.303887  7353 raft_consensus.cc:2468] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:04.306612  8335 raft_consensus.cc:697] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:04.307808  8335 consensus_queue.cc:237] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:04.311869  7504 raft_consensus.cc:1275] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.313922  8341 consensus_queue.cc:1048] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.326153  7505 raft_consensus.cc:1275] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.327646  8335 consensus_queue.cc:1048] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.328116  7178 catalog_manager.cc:5697] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:04.355281  7353 raft_consensus.cc:1275] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.356729  8341 consensus_queue.cc:1048] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.501590  7427 raft_consensus.cc:1275] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.506925  7504 raft_consensus.cc:1275] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.506522  8340 consensus_queue.cc:1048] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.517627  8341 consensus_queue.cc:1048] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.690567  7504 raft_consensus.cc:1275] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.691864  8340 consensus_queue.cc:1048] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.709024  7426 raft_consensus.cc:1275] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.711650  8367 consensus_queue.cc:1048] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.714763  7353 raft_consensus.cc:1275] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.716614  8335 consensus_queue.cc:1048] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.732030  7504 raft_consensus.cc:1275] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.736380  8335 consensus_queue.cc:1048] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.770931  7353 raft_consensus.cc:1275] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.773481  8341 consensus_queue.cc:1048] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.783869  7504 raft_consensus.cc:1275] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.793747  8341 consensus_queue.cc:1048] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:04.858237 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:04.904122 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:04.946028  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:04.946631  7504 raft_consensus.cc:606] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:04.949324  7353 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e"
dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:59012
I20260709 13:16:04.949891  7353 raft_consensus.cc:606] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:04.952605  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "343f79626b484d4aa20e247aac64c837"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:04.953022  7504 raft_consensus.cc:606] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:04.955039  7504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d"
dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:53844
I20260709 13:16:04.956346  7504 raft_consensus.cc:606] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:04.960536  7353 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf"
dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
mode: GRACEFUL
new_leader_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:59012
I20260709 13:16:04.961059  7353 raft_consensus.cc:606] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Received request to transfer leadership to TS 2b2b311085b64300925884dec9a28c7d
I20260709 13:16:04.962270 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:04.963894 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:05.126618  8328 raft_consensus.cc:993] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.127286  8349 raft_consensus.cc:1081] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.128809  8121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:05.129340  8121 raft_consensus.cc:493] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.129619  8121 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.137295  8121 raft_consensus.cc:515] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.139350  8121 leader_election.cc:290] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:05.140038  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:05.140358  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942a30e040ee4dcca90e52f6dc78a64d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:05.140915  7426 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.140974  7504 raft_consensus.cc:3055] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:05.141381  7504 raft_consensus.cc:740] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:05.142257  7504 consensus_queue.cc:260] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.143684  7504 raft_consensus.cc:3060] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.148746  7426 raft_consensus.cc:2468] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.150194  8067 leader_election.cc:304] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:05.151078  8341 raft_consensus.cc:2804] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:05.151473  7504 raft_consensus.cc:2468] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.151574  8341 raft_consensus.cc:697] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:05.152691  8341 consensus_queue.cc:237] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.172264  8336 raft_consensus.cc:993] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.175035  8332 raft_consensus.cc:1081] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.176949  8121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49082
I20260709 13:16:05.177497  8121 raft_consensus.cc:493] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.184398  8121 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.177459  7177 catalog_manager.cc:5697] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:05.188939  8328 raft_consensus.cc:993] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.189445  8328 raft_consensus.cc:1081] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.196780  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:05.197329  8115 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.197685  8115 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.198096  8121 raft_consensus.cc:515] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.200363  8121 leader_election.cc:290] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:05.200979  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:05.201635  7353 raft_consensus.cc:3055] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:05.201922  7353 raft_consensus.cc:740] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:05.202522  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:05.203117  7427 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.202766  7353 consensus_queue.cc:260] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.208364  7353 raft_consensus.cc:3060] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.228334  7353 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.228812  7427 raft_consensus.cc:2468] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.229766  8332 raft_consensus.cc:993] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.229806  8070 leader_election.cc:304] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:05.230485  8332 raft_consensus.cc:1081] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.230892  8341 raft_consensus.cc:2804] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:05.231415  8341 raft_consensus.cc:697] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:05.232527  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49082
I20260709 13:16:05.232316  8341 consensus_queue.cc:237] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.233114  8117 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.233548  8117 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.242821  8117 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.244645  8117 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:05.227137  8115 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:05.245945  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:05.246570  7430 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.247460  8115 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:05.248891  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:05.249555  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:05.249549  7357 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.250283  7504 raft_consensus.cc:3055] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:05.250197  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:05.250677  7504 raft_consensus.cc:740] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:05.251044  7353 raft_consensus.cc:3055] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:05.251451  7353 raft_consensus.cc:740] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:05.252431  7353 consensus_queue.cc:260] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.253405  7504 consensus_queue.cc:260] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:05.255206  7504 raft_consensus.cc:3060] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.258487  8328 raft_consensus.cc:993] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: : Instructing follower 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.258967  8328 raft_consensus.cc:1081] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Signalling peer 2b2b311085b64300925884dec9a28c7d to start an election
I20260709 13:16:05.263059  7357 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:05.264493  8070 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:05.265668  8341 raft_consensus.cc:2804] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:05.266181  8341 raft_consensus.cc:697] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:05.267132  8341 consensus_queue.cc:237] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:05.274652  7430 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:05.276718  8067 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:05.253859  7353 raft_consensus.cc:3060] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.277347  8115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "343f79626b484d4aa20e247aac64c837"
dest_uuid: "2b2b311085b64300925884dec9a28c7d"
 from {username='slave'} at 127.0.0.1:49092
I20260709 13:16:05.277958  8115 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.278281  8115 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.281294  8341 raft_consensus.cc:2804] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:05.285523  7504 raft_consensus.cc:2468] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:05.285662  8341 raft_consensus.cc:697] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:05.288774  7353 raft_consensus.cc:2468] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 3.
I20260709 13:16:05.289328  8341 consensus_queue.cc:237] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:05.295068  8115 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.297894  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:05.298518  7427 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.298624  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:05.299333  7504 raft_consensus.cc:3055] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:05.299645  7504 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:05.300459  7504 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.301484  7504 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:05.304205  7427 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.304512  8115 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:05.305506  8067 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:05.306728  8341 raft_consensus.cc:2804] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:05.307233  8341 raft_consensus.cc:697] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:05.308230  8341 consensus_queue.cc:237] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:05.309948  7504 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:05.311152  7177 catalog_manager.cc:5697] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.312984  7177 catalog_manager.cc:5697] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.314594  7177 catalog_manager.cc:5697] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 2 to 3, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 3 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.335155  7177 catalog_manager.cc:5697] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d reported cstate change: term changed from 1 to 2, leader changed from 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4) to 2b2b311085b64300925884dec9a28c7d (127.31.120.1). New cstate: current_term: 2 leader_uuid: "2b2b311085b64300925884dec9a28c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:05.603564  7505 raft_consensus.cc:1275] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.604943  8341 consensus_queue.cc:1048] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:05.615096  7427 raft_consensus.cc:1275] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.618350  8404 consensus_queue.cc:1048] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:05.694772  7357 raft_consensus.cc:1275] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.696393  8404 consensus_queue.cc:1048] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.698478  7357 raft_consensus.cc:1275] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.701659  8404 consensus_queue.cc:1048] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:05.721643  7427 raft_consensus.cc:1275] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.725135  8404 consensus_queue.cc:1048] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:05.725735  7427 raft_consensus.cc:1275] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.731906  8340 consensus_queue.cc:1048] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:05.770696  7505 raft_consensus.cc:1275] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.773371  8367 consensus_queue.cc:1048] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:05.795928  8070 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 81418.1us
I20260709 13:16:05.798447  7353 raft_consensus.cc:1275] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.810432  8341 consensus_queue.cc:1048] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:05.841809  7423 raft_consensus.cc:1275] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.843396  8419 consensus_queue.cc:1048] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:05.857504  7505 raft_consensus.cc:1275] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 2b2b311085b64300925884dec9a28c7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:05.858880  8404 consensus_queue.cc:1048] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:05.964792 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:06.016127 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:06.055766  7357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb"
dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
mode: GRACEFUL
new_leader_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:59012
I20260709 13:16:06.056411  7357 raft_consensus.cc:606] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Received request to transfer leadership to TS 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:06.057782 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:16:06.058960 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:06.180614  8336 raft_consensus.cc:993] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: : Instructing follower 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:06.181154  8332 raft_consensus.cc:1081] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Signalling peer 5270a004873440ec9a5e8106b63a6a85 to start an election
I20260709 13:16:06.182722  7426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb"
dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
 from {username='slave'} at 127.0.0.1:40226
I20260709 13:16:06.183236  7426 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:06.183548  7426 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.189628  7426 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:06.192977  7426 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:06.193890  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:06.194401  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:06.195214  7352 raft_consensus.cc:3055] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:06.194536  7504 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.195695  7352 raft_consensus.cc:740] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:06.196587  7352 consensus_queue.cc:260] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:06.197973  7352 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.201927  7504 raft_consensus.cc:2468] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:06.203240  7394 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:06.204105  8370 raft_consensus.cc:2804] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:06.204636  8370 raft_consensus.cc:697] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:06.205538  8370 consensus_queue.cc:237] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:06.206406  7352 raft_consensus.cc:2468] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:06.217027  7179 catalog_manager.cc:5697] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:06.680480  7504 raft_consensus.cc:1275] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:06.681917  8411 consensus_queue.cc:1048] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:06.698414  7352 raft_consensus.cc:1275] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:06.702428  8370 consensus_queue.cc:1048] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:07.059885 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:07.136439 32224 tablet_server.cc:179] TabletServer@127.31.120.1:33107 shutting down...
W20260709 13:16:07.142699  7468 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 152 similar messages]
W20260709 13:16:07.148620  7468 consensus_peers.cc:597] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.172065 32224 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 8126)
I20260709 13:16:08.205013  7352 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49830 (request call id 58) took 1043 ms. Trace:
W20260709 13:16:08.205034  7211 debug-util.cc:398] Leaking SignalData structure 0x7b0800363820 after lost signal to thread 32227
I20260709 13:16:08.205639  7352 rpcz_store.cc:276] 0709 13:16:07.161624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.204150 (+1042526us) service_pool.cc:224] Handling call
0709 13:16:08.204491 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.204683 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.204710 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:08.204752 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:08.204780 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.204834 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.204954 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:08.210541  7320 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.011s	sys 0.002s
W20260709 13:16:08.211938  7320 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.013s	sys 0.003s
W20260709 13:16:08.212387  7320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 1.05251e+06us
I20260709 13:16:08.222483  8450 raft_consensus.cc:493] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.229586  8447 raft_consensus.cc:493] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.244881  8447 raft_consensus.cc:515] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
W20260709 13:16:08.240878  8069 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.004s	sys 0.013s
I20260709 13:16:08.247566  8447 leader_election.cc:290] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
W20260709 13:16:08.248369  8069 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.005s	sys 0.014s
I20260709 13:16:08.248418  8450 raft_consensus.cc:515] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.251961  8341 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.252444  8341 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.254446  8341 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.262552  8450 leader_election.cc:290] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.268412  8438 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.272370  8448 raft_consensus.cc:493] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.339210  8438 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.339286  8448 raft_consensus.cc:515] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.341393  8438 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.341648  8448 leader_election.cc:290] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.342995  8448 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.299118  8367 raft_consensus.cc:493] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.346374  8367 raft_consensus.cc:515] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.348701  8367 leader_election.cc:290] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.350080  8457 raft_consensus.cc:493] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
W20260709 13:16:08.305146  7211 debug-util.cc:398] Leaking SignalData structure 0x7b08003dcee0 after lost signal to thread 7214
I20260709 13:16:08.350585  8457 raft_consensus.cc:515] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.350961  8448 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.351915  8453 raft_consensus.cc:493] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
W20260709 13:16:08.352164  7211 debug-util.cc:398] Leaking SignalData structure 0x7b0800113320 after lost signal to thread 7381
I20260709 13:16:08.352519  8453 raft_consensus.cc:515] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
W20260709 13:16:08.353864  7211 debug-util.cc:398] Leaking SignalData structure 0x7b08000f38c0 after lost signal to thread 7455
I20260709 13:16:08.354115  8448 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.355157  8453 leader_election.cc:290] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.312522  8445 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.314203  8455 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.322028  8328 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.325131  8454 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.329770  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.335494  8441 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.271453  8436 raft_consensus.cc:493] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.308571  8456 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.360200  8328 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.385233  8453 raft_consensus.cc:493] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.385914  8453 raft_consensus.cc:515] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.388211  8453 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.401278  8332 raft_consensus.cc:493] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.435286  8450 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.436417  8450 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.357707  8446 raft_consensus.cc:493] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.363137  8448 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.442608  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:08.355547  7211 debug-util.cc:398] Leaking SignalData structure 0x7b0800074680 after lost signal to thread 7529
I20260709 13:16:08.390847  8341 raft_consensus.cc:493] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
W20260709 13:16:08.445755  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 115008us
I20260709 13:16:08.358393  8452 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
W20260709 13:16:08.447368  8067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 101533us
I20260709 13:16:08.433516  8332 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.357234  8457 leader_election.cc:290] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.361054  8461 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.439219  8450 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.364771  8468 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.358505  8336 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.449887  8341 raft_consensus.cc:515] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.430495  8367 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.468459  8473 raft_consensus.cc:493] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.468276  8464 raft_consensus.cc:493] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.362619  8441 raft_consensus.cc:515] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.363209  8456 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.481927  8465 raft_consensus.cc:493] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.444535  8448 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.473029  8370 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.473459  8457 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.359514  8445 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.362865  8436 raft_consensus.cc:515] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.359616  8455 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.532548  8328 leader_election.cc:290] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
W20260709 13:16:08.497825  7211 debug-util.cc:398] Leaking SignalData structure 0x7b08000631a0 after lost signal to thread 8183
I20260709 13:16:08.440074  8446 raft_consensus.cc:515] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.559310  8341 leader_election.cc:290] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.508473  8450 raft_consensus.cc:493] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.561272  8328 raft_consensus.cc:493] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.484340  8453 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.359997  8454 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.562608  8332 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.563433  8328 raft_consensus.cc:515] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.566690  8341 raft_consensus.cc:493] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.567257  8341 raft_consensus.cc:515] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.470731  8452 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.476826  8461 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.581557  8332 raft_consensus.cc:493] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.535414  8367 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.587491  8341 leader_election.cc:290] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.588119  8341 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.560882  8465 raft_consensus.cc:515] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.562960  8457 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.591094  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.495270  8336 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.591719  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.592427  7501 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:08.570425  8451 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.590237  8367 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.596263  8341 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.597541  8332 raft_consensus.cc:515] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.494606  8464 raft_consensus.cc:515] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.593300  8447 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.602680  8456 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.602545  8447 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.602869  8469 raft_consensus.cc:493] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.603181  8464 leader_election.cc:290] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.603339  8463 raft_consensus.cc:493] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.603444  8469 raft_consensus.cc:515] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.603981  8463 raft_consensus.cc:515] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.604647  8447 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.494899  8473 raft_consensus.cc:515] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.605027  8470 raft_consensus.cc:493] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.605522  8470 raft_consensus.cc:515] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.606297  8463 leader_election.cc:290] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.606732  8473 leader_election.cc:290] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.569074  8453 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.607971  8436 leader_election.cc:290] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.613575  8328 leader_election.cc:290] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.494202  8468 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.592764  8457 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.625200  8457 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.625573  8446 leader_election.cc:290] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.625717  8457 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.592600  8465 leader_election.cc:290] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.627489  8461 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.628830  8454 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.631229  8478 raft_consensus.cc:493] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.631533  8452 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.621177  8449 raft_consensus.cc:493] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.632118  8452 raft_consensus.cc:493] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.631970  8478 raft_consensus.cc:515] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.632232  8449 raft_consensus.cc:515] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.632695  8452 raft_consensus.cc:515] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.621445  8341 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.621891  8467 raft_consensus.cc:493] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.634673  8478 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.634536  8467 raft_consensus.cc:515] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.636689  8467 leader_election.cc:290] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.622030  8466 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.622764  8336 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.593065  8459 raft_consensus.cc:493] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.637871  8466 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.638238  8459 raft_consensus.cc:515] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.639158  8452 leader_election.cc:290] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.639858  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.640148  8466 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.640506  8459 leader_election.cc:290] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.640579  7497 raft_consensus.cc:2468] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
I20260709 13:16:08.640990  8332 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.623023  8441 leader_election.cc:290] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.623842  8473 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.568567  8450 raft_consensus.cc:515] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.641650  8473 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.643113  8450 leader_election.cc:290] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.570744  8370 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.643885  8438 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.644425  8438 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.648947  8455 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.649363  8453 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.619710  8463 raft_consensus.cc:493] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.650712  8463 raft_consensus.cc:515] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.645366  8370 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.652982  8463 leader_election.cc:290] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.655045  8468 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.646108  8473 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.647861  8457 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.677042  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.612308  8471 raft_consensus.cc:493] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.635327  8478 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.612746  8470 leader_election.cc:290] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.656610  8449 leader_election.cc:290] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
W20260709 13:16:08.691524  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 103569us
I20260709 13:16:08.635365  8445 leader_election.cc:290] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.656634  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.685192  8473 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.701300  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.701924  7426 raft_consensus.cc:2468] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.685948  8468 raft_consensus.cc:493] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.705912  8341 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.690603  8459 raft_consensus.cc:493] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.706364  8341 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.612999  8451 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.696363  8467 raft_consensus.cc:493] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.731206  7394 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:08.704800  8470 raft_consensus.cc:493] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.706243  8441 raft_consensus.cc:493] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.697654  8478 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.717038  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.715814  8445 raft_consensus.cc:493] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.731987  8438 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.741804  7394 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:08.744258  8341 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.744880  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aa90396c8ee443fa7e1f1e7308e594e" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.743129  8487 raft_consensus.cc:493] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.729681  8468 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.745926  8487 raft_consensus.cc:515] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.746331  8451 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.747376  8447 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.747895  8492 raft_consensus.cc:493] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.748093  8447 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.748425  8492 raft_consensus.cc:515] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.749562  8478 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.750157  8445 raft_consensus.cc:515] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.750589  8492 leader_election.cc:290] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.736867  8481 raft_consensus.cc:493] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.751303  8472 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.751682  8481 raft_consensus.cc:515] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.751829  8472 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.752452  8445 leader_election.cc:290] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.753794  8481 leader_election.cc:290] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.703368  8471 raft_consensus.cc:515] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.754797  8445 raft_consensus.cc:493] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.755200  8367 raft_consensus.cc:493] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.743335  8465 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.755775  8367 raft_consensus.cc:515] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.756517  8465 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.758625  8465 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.747464  8449 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:08.759622  8449 raft_consensus.cc:493] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.759989  8449 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.760038  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.761791  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.762001  7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.747635  8474 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.763176  8474 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.743232  8461 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.741802  8470 raft_consensus.cc:515] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.743132  8467 raft_consensus.cc:515] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.764321  8461 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.765658  7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.728272  8473 raft_consensus.cc:515] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.766690  8461 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.750530  8489 raft_consensus.cc:493] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.747452  8468 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.766074  8467 leader_election.cc:290] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.783293  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.784334  7352 raft_consensus.cc:2468] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.763583  8367 leader_election.cc:290] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.803947  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad0292559534756b194428330c7c717" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.806026  8469 leader_election.cc:290] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.790822  8465 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.806717  8488 raft_consensus.cc:493] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.795931  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.756021  8471 leader_election.cc:290] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.736532  8459 raft_consensus.cc:515] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.797372  7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.796684  7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.809267  7349 raft_consensus.cc:2468] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:08.809660  8459 leader_election.cc:290] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.759816  7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.800712  8494 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.801839  8475 raft_consensus.cc:493] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.769335  8473 leader_election.cc:290] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.810456  7350 raft_consensus.cc:2468] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:08.802830  8487 leader_election.cc:290] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.774323  8438 raft_consensus.cc:2804] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:08.807148  8465 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.811465  8438 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:08.807327  8488 raft_consensus.cc:515] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.811928  8438 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.813477  8488 leader_election.cc:290] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.813671  8465 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.810438  8494 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.814561  8487 raft_consensus.cc:493] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.802805  8448 leader_election.cc:290] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.801281  8474 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.750177  8447 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.815053  8487 raft_consensus.cc:515] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.742249  8441 raft_consensus.cc:515] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.816329  8367 raft_consensus.cc:493] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.816932  8367 raft_consensus.cc:515] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:08.789608  8468 raft_consensus.cc:493] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.787653  8489 raft_consensus.cc:515] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.818353  8468 raft_consensus.cc:515] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.820279  8468 leader_election.cc:290] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.766074  8470 leader_election.cc:290] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.753794  8472 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.817762  8441 leader_election.cc:290] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.815809  8494 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.823940  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.819383  8367 leader_election.cc:290] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.828277  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abce17e47154429b9f10b822f48ab3e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.828635  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.830073  7425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bce1894cd442d5bc2580cb8d02012d" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.829057  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99f8dd60683a4a94b2eab776a941327a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.831331  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.829434  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.755348  8445 raft_consensus.cc:515] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.833554  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad9e725766854f9f8f7ab7741e09179b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.834100  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.834676  8445 leader_election.cc:290] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.834743  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.835173  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.836063  7425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.836483  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.810504  8475 raft_consensus.cc:515] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.837239  7355 raft_consensus.cc:2468] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.838400  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76381a75f921496eb6274bd7df95f601" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.839119  8475 leader_election.cc:290] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.839293  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.839784  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3fe3d05909404e9c7e64e0f20ba873" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.839965  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.841013  7499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.841673  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.842123  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.817770  8487 leader_election.cc:290] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.843047  7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.843860  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.844338  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.845425  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "239bdb3893f6422a9456da1eec09ac9e" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.846167  7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57da519e6ecb4abbaf3b5361a4e0b9ee" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.834749  7423 raft_consensus.cc:2468] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.847216  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.847506  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961fad27981341bb99577079ea5dcc3f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.848057  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.848476  7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc1db9e93f304836b7e9f3bc59c89600" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.848970  7354 raft_consensus.cc:2468] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:08.850440  8449 raft_consensus.cc:515] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.842048  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.855899  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.856280  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.856506  7505 raft_consensus.cc:2468] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 1.
I20260709 13:16:08.857585  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.820295  8489 leader_election.cc:290] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.864012  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f035dcc1e4cc4e6fae948203b417e7cd" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.865381  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.866086  7348 raft_consensus.cc:2468] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:16:08.866595  8449 leader_election.cc:290] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:08.867625  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.868278  7348 raft_consensus.cc:2468] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 2.
I20260709 13:16:08.869637  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.872325  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.874153  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.874747  7348 raft_consensus.cc:2468] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:08.874643  8438 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:08.876871  8438 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:08.878232  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a88676349042cda17ab04278fa8b6f" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.878652  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.919477  7354 raft_consensus.cc:2468] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49f7f0d3dec548bbb5538d2b1ddb4415 in term 1.
I20260709 13:16:08.919673  8070 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:08.919831  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.920442  8445 raft_consensus.cc:2804] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:08.920442  7354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.878902  7348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.921006  8445 raft_consensus.cc:493] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.921337  7354 raft_consensus.cc:2468] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.921468  8445 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.879894  7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.921531  7471 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:08.922140  7349 raft_consensus.cc:2468] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.922537  8482 raft_consensus.cc:2804] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:08.880115  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.880623  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.880714  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.880802  7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.882954  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.884721  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.885381  7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:08.886400  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.888233  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:16:08.890319  7320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 158305us
I20260709 13:16:08.891952  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eb1ec31ba3340fb815cc11f5c116aa0" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.896126 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:08.914111  8487 raft_consensus.cc:493] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:08.930647  8487 raft_consensus.cc:515] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.931749  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.932751  8487 leader_election.cc:290] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.914893  7468 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:08.933990  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.934285  7468 leader_election.cc:304] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:08.935032  8328 raft_consensus.cc:2749] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:08.935284  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.935696  8328 raft_consensus.cc:2749] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.936896  8070 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:08.922948  8482 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.938038 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:08.938258  8482 raft_consensus.cc:3060] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.938503  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2118345d07be4c3195f476caf906740a" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.939268 32224 raft_consensus.cc:2243] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.939931 32224 raft_consensus.cc:2272] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.939867  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc3465e298174cd9b6cdf4a7ea730ebe" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.922706  7394 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85; no voters: 
I20260709 13:16:08.923185  7352 raft_consensus.cc:2468] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 1.
I20260709 13:16:08.942972  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:08.944468  7468 consensus_peers.cc:597] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.944830  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:16:08.945115  7468 consensus_peers.cc:597] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.945699  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.946059  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.947319  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04afbc5d731346f9a3c99d9d89f097f0" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.947736  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.948942  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.950093  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.950397  7426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.937927  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:08.951037  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2b493283374c639936c29e00cdd96c" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.950436  7425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.937364  8445 raft_consensus.cc:515] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:08.952486  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.952659  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.953428  8445 leader_election.cc:290] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:08.953866  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.954051  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.955006  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.955371  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eac7e41e3a46a2aed7ecb29109d99d" candidate_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:08.945719  7468 consensus_peers.cc:597] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.957132  7468 consensus_peers.cc:597] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.957751  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f21194252b0411c9c67f78fcfec3140" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.957799  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e534925d6a411990fea3bbc14a4935" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:08.963181  8497 raft_consensus.cc:2804] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:08.963564  8497 raft_consensus.cc:493] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:08.963932  8497 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:08.966568  7394 consensus_peers.cc:597] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.918426  8067 leader_election.cc:304] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.969425  7320 leader_election.cc:304] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.970381  8454 raft_consensus.cc:2749] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:08.970014  8067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 51672.7us
I20260709 13:16:08.971760  8489 raft_consensus.cc:2749] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.972312  7320 leader_election.cc:304] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.973043  8454 raft_consensus.cc:2749] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:08.973476  8067 leader_election.cc:304] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:08.974225  8487 raft_consensus.cc:2749] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.974457  7320 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.975220  8454 raft_consensus.cc:2749] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.975934  7471 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:08.976382  7320 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.976644  8328 raft_consensus.cc:2804] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:08.977063  8328 raft_consensus.cc:493] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:08.977104  8454 raft_consensus.cc:2749] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.947986 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:08.977491  8328 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.978502  7320 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.979212  8454 raft_consensus.cc:2749] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.979171  8482 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.979579  7320 leader_election.cc:304] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:08.980370  8454 raft_consensus.cc:2749] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:08.982028  7394 consensus_peers.cc:597] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.983541  8328 raft_consensus.cc:515] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:08.985270  8328 leader_election.cc:290] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:08.986840 32224 raft_consensus.cc:2243] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:16:08.957697  7468 consensus_peers.cc:597] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.990757  7468 consensus_peers.cc:597] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.991355  7468 consensus_peers.cc:597] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.991906  7468 consensus_peers.cc:597] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.992442  7468 consensus_peers.cc:597] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.995630  7468 consensus_peers.cc:597] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.996217  7468 consensus_peers.cc:597] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.996786  7468 consensus_peers.cc:597] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.997277  7468 leader_election.cc:336] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:08.997673  7468 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:08.998334  8328 raft_consensus.cc:2749] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:08.998693  7468 leader_election.cc:336] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:08.999197  7468 leader_election.cc:304] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.001885  8070 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:35887, user_credentials={real_user=slave}} blocked reactor thread for 65058.4us
I20260709 13:16:09.001995  8487 raft_consensus.cc:2804] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.002521  8487 raft_consensus.cc:493] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:09.002641  8070 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.002931  8487 raft_consensus.cc:3060] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.003507  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.003835  8070 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.004676  8489 raft_consensus.cc:2804] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.004911  8070 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.005075  8489 raft_consensus.cc:493] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49f7f0d3dec548bbb5538d2b1ddb4415)
I20260709 13:16:09.005507  8489 raft_consensus.cc:3060] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.005681  8472 raft_consensus.cc:2804] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.006129  8472 raft_consensus.cc:493] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.006501  8472 raft_consensus.cc:3060] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.010282  7430 raft_consensus.cc:1240] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Rejecting Update request from peer 71e04cb9c24b4ddd9d56199a77f5aa89 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:09.010303 32224 raft_consensus.cc:2272] T 56bce1894cd442d5bc2580cb8d02012d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.011154  8487 raft_consensus.cc:515] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:09.012171  8070 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.013002  8488 raft_consensus.cc:2804] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.013377  8488 raft_consensus.cc:493] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.013406  8487 leader_election.cc:290] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:09.013923  8488 raft_consensus.cc:3060] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.014218  8487 raft_consensus.cc:2804] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.014675  8487 raft_consensus.cc:493] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.015002 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.015089  8487 raft_consensus.cc:3060] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.015789  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0079a4cd3d134d079f559b4073824860" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.016281 32224 raft_consensus.cc:2243] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.017328 32224 raft_consensus.cc:2272] T dc1db9e93f304836b7e9f3bc59c89600 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.017973  8067 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.020200  8070 leader_election.cc:304] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
W20260709 13:16:09.020996  7394 consensus_peers.cc:597] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.030462  7394 consensus_peers.cc:597] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.030547  8070 leader_election.cc:304] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.021783  8482 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.031911  8367 raft_consensus.cc:2804] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.032336  8367 raft_consensus.cc:493] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.032795  8367 raft_consensus.cc:3060] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.034225 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.022516  8487 raft_consensus.cc:515] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.023311  8472 raft_consensus.cc:515] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.034994  8070 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.032323  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec7093b8e1740cba58723ac7119486b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.028206  8469 consensus_queue.cc:1059] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:09.036075  8472 leader_election.cc:290] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.036864  8469 raft_consensus.cc:3055] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:09.037272  8469 raft_consensus.cc:740] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:09.037431 32224 raft_consensus.cc:2243] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.038092 32224 raft_consensus.cc:2272] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.038177  8469 consensus_queue.cc:260] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
W20260709 13:16:09.039709  8454 consensus_queue.cc:1175] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260709 13:16:09.041363 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.036110  8487 leader_election.cc:290] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.029604  8445 raft_consensus.cc:2749] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.030550  8496 raft_consensus.cc:2804] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.043286 32224 raft_consensus.cc:2243] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.042980  8367 raft_consensus.cc:515] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.043617  8496 raft_consensus.cc:493] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.043991 32224 raft_consensus.cc:2272] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.044106  8496 raft_consensus.cc:3060] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.045657  8367 leader_election.cc:290] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.051496  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271a5a5acaf443f6ae8e2122f6f16166" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.053043  7355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.053414  7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.026298  8488 raft_consensus.cc:515] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.035835  8468 raft_consensus.cc:2804] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.054247  8468 raft_consensus.cc:493] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.054639  8468 raft_consensus.cc:3060] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.055249  8497 raft_consensus.cc:515] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.047495 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.049031  8482 raft_consensus.cc:2749] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:09.049139  7468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.1:33107, user_credentials={real_user=slave}} blocked reactor thread for 50468.8us
I20260709 13:16:09.049919  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce24f22b1cbe4955a186eb1a28aa32bf" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
W20260709 13:16:09.057803  7468 leader_election.cc:336] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.051910  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.058211  7468 leader_election.cc:304] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.014614  8489 raft_consensus.cc:515] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:09.052560  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e82c6ed54a49e39710ec850e7ac826" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.058991  8482 raft_consensus.cc:2749] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.055565  8488 leader_election.cc:290] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.047986  8067 leader_election.cc:304] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.060397  8488 raft_consensus.cc:2749] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.039542  8469 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.060940  8489 leader_election.cc:290] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Requested vote from peers 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:09.061774  7471 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.069586  7397 leader_election.cc:304] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.070106 32224 raft_consensus.cc:2243] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.064188  8482 raft_consensus.cc:2804] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.070758  8449 raft_consensus.cc:2749] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.070883  8482 raft_consensus.cc:493] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5270a004873440ec9a5e8106b63a6a85)
I20260709 13:16:09.071492  8482 raft_consensus.cc:3060] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.073385  7430 raft_consensus.cc:1240] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Rejecting Update request from peer 71e04cb9c24b4ddd9d56199a77f5aa89 for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:09.062531  7468 leader_election.cc:336] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.078028  7468 leader_election.cc:304] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.075263  8067 leader_election.cc:304] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.079247  8328 raft_consensus.cc:2749] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.079893  8489 raft_consensus.cc:2804] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.080360  8489 raft_consensus.cc:493] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 71e04cb9c24b4ddd9d56199a77f5aa89)
I20260709 13:16:09.080796  8489 raft_consensus.cc:3060] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.082333  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" is_pre_election: true
I20260709 13:16:09.083717  8486 consensus_queue.cc:1059] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.084276  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.084894  7357 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.085033  8454 raft_consensus.cc:3055] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:09.085541  8454 raft_consensus.cc:740] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 71e04cb9c24b4ddd9d56199a77f5aa89, State: Running, Role: LEADER
I20260709 13:16:09.086500  8454 consensus_queue.cc:260] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.087931 32224 raft_consensus.cc:2272] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.088376  8454 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.092334  7357 raft_consensus.cc:2468] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:09.093355  8067 leader_election.cc:304] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.093560  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:16:09.079270  7468 leader_election.cc:336] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.094307  7397 leader_election.cc:304] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.097590  7468 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.098377  8328 raft_consensus.cc:2749] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:09.098744  7468 leader_election.cc:336] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.099028  8449 raft_consensus.cc:2749] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.099201  7468 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.099684  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
W20260709 13:16:09.100375  7468 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.100665  8328 raft_consensus.cc:2749] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.100827  7468 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.095017  8497 leader_election.cc:290] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 49f7f0d3dec548bbb5538d2b1ddb4415 (127.31.120.4:39007)
I20260709 13:16:09.095654  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.097097  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.102435  8328 raft_consensus.cc:2749] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.102906  8488 raft_consensus.cc:2749] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.103188  7353 raft_consensus.cc:1240] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Rejecting Update request from peer 49f7f0d3dec548bbb5538d2b1ddb4415 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:09.104348  8067 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.105103  8488 raft_consensus.cc:2749] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.105291  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.106261  8067 leader_election.cc:304] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.091380 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.097064  8468 raft_consensus.cc:515] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.107511  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.110384  8482 raft_consensus.cc:515] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.073462  8496 raft_consensus.cc:515] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.112286  8328 consensus_queue.cc:1059] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:09.113230  8496 leader_election.cc:290] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.113247  8328 raft_consensus.cc:3055] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:09.113757  8328 raft_consensus.cc:740] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49f7f0d3dec548bbb5538d2b1ddb4415, State: Running, Role: LEADER
I20260709 13:16:09.107834 32224 raft_consensus.cc:2243] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.108312  8488 raft_consensus.cc:2749] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.108537  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.114485  8328 consensus_queue.cc:260] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:09.108122  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.115540 32224 raft_consensus.cc:2272] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.116012  8328 raft_consensus.cc:3060] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.118825 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.109365  8067 leader_election.cc:304] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.120035  8488 raft_consensus.cc:2749] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.121846  8067 leader_election.cc:304] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85; no voters: 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.122476  8067 raft_consensus.cc:2686] T 46e82c6ed54a49e39710ec850e7ac826 P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:09.124106  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.126036  7394 leader_election.cc:304] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.127759  8491 raft_consensus.cc:2749] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.129937  8482 leader_election.cc:290] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.111011  8468 leader_election.cc:290] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.133790  7350 raft_consensus.cc:2468] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 3.
I20260709 13:16:09.138768  7357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.140097  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.140267  7353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.114259  8489 raft_consensus.cc:515] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.142498  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.145381  7356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.146878  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.147202 32224 raft_consensus.cc:2243] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.148416  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d76f633b67148ef905cfa3b1d5c590e" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:09.150466  7394 leader_election.cc:336] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.155543  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.157836  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343f79626b484d4aa20e247aac64c837" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.159030  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af02e3c522a4048b817ec444a3ad5e6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.153635  8070 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
W20260709 13:16:09.152758  7320 consensus_peers.cc:597] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.155798  7394 consensus_peers.cc:597] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.165347  7320 leader_election.cc:336] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.168193  7320 leader_election.cc:304] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.166790  7397 leader_election.cc:304] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
W20260709 13:16:09.166442  7394 leader_election.cc:336] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.165690 32224 raft_consensus.cc:2272] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.169919  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bbbbedcee0446abe898050f5dd1578" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.170511  7394 leader_election.cc:304] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.173702  7468 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.175145  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e566e4da5c4b478489086154cbbd13" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.175788  8491 raft_consensus.cc:2749] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.176350  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "623d8610e992488e9302c20596f4a302" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.177516  7501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141286c286284003b795291ad8766165" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.179811  7503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.179891  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5fcfd6d243e5940a37ab978b4f2b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.180183  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178ebd1ec9704b868e8dec3ccdfad30d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.181207  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad3d31ca56b45dda55d0616cc89204b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.182032  7503 raft_consensus.cc:1240] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:09.182317  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd43c082a974fbe87828b19abd5bc02" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.183303  7504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a84c07d08e47da9a71cd1c513c6105" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.184691  8438 raft_consensus.cc:2804] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:09.185210  8438 raft_consensus.cc:697] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:09.185715  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.186030  8438 consensus_queue.cc:237] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.163897  7424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.188210  8458 raft_consensus.cc:2749] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.170161  7425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.152145  7497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a1a55fbf26d4f84bafaab065d4feece" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.170158  8488 raft_consensus.cc:2804] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:09.175318  8454 raft_consensus.cc:2749] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.191315  7505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa22447b4d024658b6470d416092c4cc" candidate_uuid: "5270a004873440ec9a5e8106b63a6a85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415"
I20260709 13:16:09.193049  7499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.194625  7502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "598d8f846ffc44fcaca77f1973eb6b7b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
W20260709 13:16:09.197597  7470 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:09.174459  7394 consensus_peers.cc:597] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.173404  7320 consensus_peers.cc:597] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.200502  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b8f1e9ac0b246e5b46bd9d85336d5eb" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:16:09.200920  7320 consensus_peers.cc:597] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.201465  7320 consensus_peers.cc:597] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.202293  8067 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.199399  7394 consensus_peers.cc:597] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.203621  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b3b562b9fd453ab8b640d44359747a" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:09.203869  7320 consensus_peers.cc:597] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.169238  7500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeedebd8085f46878c1a27393ea26f13" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" is_pre_election: true
I20260709 13:16:09.183461  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8ac1115f84af29bb612331df632a6" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
I20260709 13:16:09.200913  7352 raft_consensus.cc:2468] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5270a004873440ec9a5e8106b63a6a85 in term 2.
W20260709 13:16:09.203764  7394 leader_election.cc:336] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:09.205495  7320 consensus_peers.cc:597] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.208597  7394 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.209002  7320 consensus_peers.cc:597] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.209518  7320 leader_election.cc:336] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.209498  7397 leader_election.cc:304] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 
I20260709 13:16:09.210139  7320 leader_election.cc:304] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.210176  8067 leader_election.cc:304] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.202405  8488 raft_consensus.cc:697] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Leader. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:09.212702  8489 leader_election.cc:290] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Requested vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:09.212541  8488 consensus_queue.cc:237] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:09.214190  7431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270e9deedc904538b2b42381a2aa3889" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.214689  7429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92a5d8ff52e4a238b88b5dd6ed2317d" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.215359  7427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0584005e4e4271a39fc114468c717b" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
W20260709 13:16:09.217454  7320 leader_election.cc:336] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.217890  7320 leader_election.cc:304] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.216190  7394 consensus_peers.cc:597] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.218931  7320 leader_election.cc:336] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:09.221073  7394 leader_election.cc:336] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.221880  7394 leader_election.cc:304] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.221401  7320 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.222784  8070 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.223899  7320 consensus_peers.cc:597] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.223990  7394 leader_election.cc:336] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.224485  7394 leader_election.cc:304] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.206221  7496 raft_consensus.cc:1240] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:09.220481  7428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71872601e484b06882011698f8b3b52" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85" is_pre_election: true
W20260709 13:16:09.224583  7320 leader_election.cc:336] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:09.225440  7468 leader_election.cc:336] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.226768  7320 leader_election.cc:304] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.226486  8493 raft_consensus.cc:2749] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.215996  8491 raft_consensus.cc:2749] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.219623  8469 raft_consensus.cc:2749] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.227921  8489 raft_consensus.cc:2749] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.215824  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd6aaff170d478f8d6920b5849c6978" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.210857  8496 raft_consensus.cc:2749] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.226950  7468 leader_election.cc:304] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.232666  7430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79d0d3ecc64a7a96c61be7604e6753" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.233098  8328 raft_consensus.cc:2749] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:09.228209  8067 raft_consensus.cc:2686] T 27a84c07d08e47da9a71cd1c513c6105 P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:16:09.233969  7468 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.234248  7352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89"
I20260709 13:16:09.234648  7468 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.235021  8067 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.216248  8449 raft_consensus.cc:2804] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:09.228662  7425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8365f37a3b4cdaa6fdfd3fb9f9e35f" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.229292  8477 raft_consensus.cc:2749] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.220928  8486 raft_consensus.cc:2749] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.240890  8328 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:09.243474  7468 leader_election.cc:336] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.244131  7468 leader_election.cc:304] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.245010  7320 leader_election.cc:336] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.245519  7320 leader_election.cc:304] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.225553  8448 raft_consensus.cc:2749] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.225375  8497 raft_consensus.cc:2749] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:09.256306  7320 leader_election.cc:336] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:09.246228  8067 raft_consensus.cc:2686] T 5a1a55fbf26d4f84bafaab065d4feece P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:16:09.245627  7468 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.247133  8449 raft_consensus.cc:697] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Leader. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
W20260709 13:16:09.241961  7394 consensus_peers.cc:597] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.256825  7320 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.261183  7394 leader_election.cc:336] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.261580  7394 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.251484 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.259706  7468 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.260342  8067 leader_election.cc:304] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.246245  8498 raft_consensus.cc:2749] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.272818  8328 raft_consensus.cc:2749] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.272169  8468 raft_consensus.cc:2749] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.272686  8067 leader_election.cc:304] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.259301  7501 raft_consensus.cc:2468] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2b311085b64300925884dec9a28c7d in term 2.
I20260709 13:16:09.273604  8477 raft_consensus.cc:2749] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:09.278810  7394 leader_election.cc:336] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.279209  7394 leader_election.cc:304] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.280458  7394 leader_election.cc:336] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.280835  7394 leader_election.cc:304] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
W20260709 13:16:09.274497  7320 consensus_peers.cc:597] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.273944  8449 consensus_queue.cc:237] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.255697  8454 raft_consensus.cc:2749] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:09.276086  7468 leader_election.cc:336] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:09.283402  8067 raft_consensus.cc:2686] T aeedebd8085f46878c1a27393ea26f13 P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:16:09.282307  7320 leader_election.cc:336] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.290834  7468 leader_election.cc:304] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.291174  7320 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.292446  7468 leader_election.cc:336] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.292824  7468 leader_election.cc:304] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.293246  7320 leader_election.cc:336] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.293679  7320 leader_election.cc:304] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
W20260709 13:16:09.294605  7320 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.294967  7320 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.297015  8328 raft_consensus.cc:2749] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.297174  7468 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.298192  8067 leader_election.cc:304] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.299039 32224 raft_consensus.cc:2243] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:09.300065  7320 leader_election.cc:336] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.300962  8458 raft_consensus.cc:2749] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.279954  8497 raft_consensus.cc:2749] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.281831  8448 raft_consensus.cc:2749] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.281961  8491 raft_consensus.cc:2749] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.298187  8498 raft_consensus.cc:2749] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.299777  8457 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.296283  8493 raft_consensus.cc:2749] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.297959  8469 raft_consensus.cc:2749] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.300592  7320 leader_election.cc:304] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.309527 32224 raft_consensus.cc:2272] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.308342  7468 leader_election.cc:304] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.311203  8328 raft_consensus.cc:2749] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:09.311336  7320 leader_election.cc:336] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.311944  7320 leader_election.cc:304] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.313563  8067 leader_election.cc:304] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.313905  7320 leader_election.cc:336] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.314633  8477 raft_consensus.cc:2749] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.314489  7320 leader_election.cc:304] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 71e04cb9c24b4ddd9d56199a77f5aa89; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.315402  8454 raft_consensus.cc:2749] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.316159  8478 raft_consensus.cc:2749] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.306491  8468 raft_consensus.cc:2749] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:09.325285  8067 raft_consensus.cc:2686] T c86b5fcfd6d243e5940a37ab978b4f2b P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:09.327798  8067 leader_election.cc:304] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.331112  8067 raft_consensus.cc:2686] T 178ebd1ec9704b868e8dec3ccdfad30d P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:09.332486  8067 leader_election.cc:304] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
W20260709 13:16:09.333989  8067 raft_consensus.cc:2686] T 9ad3d31ca56b45dda55d0616cc89204b P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:09.338943  8067 leader_election.cc:304] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.340010  8468 raft_consensus.cc:2749] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.341285  8067 leader_election.cc:304] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.342005  8468 raft_consensus.cc:2749] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.344594  8067 leader_election.cc:304] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.345495  8496 raft_consensus.cc:2749] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.346880  8067 leader_election.cc:304] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.347671  8496 raft_consensus.cc:2749] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.347836 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.348745 32224 raft_consensus.cc:2243] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.348930  8067 leader_election.cc:304] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.349260 32224 raft_consensus.cc:2272] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.352319 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.353216 32224 raft_consensus.cc:2243] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.353847 32224 raft_consensus.cc:2272] T 8af02e3c522a4048b817ec444a3ad5e6 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.354444  8496 raft_consensus.cc:2749] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.355335  8067 leader_election.cc:304] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.364233  8067 leader_election.cc:304] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.365705  8067 leader_election.cc:304] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
W20260709 13:16:09.367579  8067 raft_consensus.cc:2686] T de79d0d3ecc64a7a96c61be7604e6753 P 2b2b311085b64300925884dec9a28c7d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:09.368906  8067 leader_election.cc:304] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.366647  8496 raft_consensus.cc:2749] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.369987  8446 raft_consensus.cc:2749] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.375700  8067 leader_election.cc:304] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 49f7f0d3dec548bbb5538d2b1ddb4415, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:09.376583  8446 raft_consensus.cc:2749] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.367914  8468 raft_consensus.cc:2749] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:09.385388  7394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.4:39007, user_credentials={real_user=slave}} blocked reactor thread for 88139.7us
I20260709 13:16:09.387503 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.395412  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ae5b6a026546828ed9671f9ca41aa0" candidate_uuid: "2b2b311085b64300925884dec9a28c7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.396919  7423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c47eeb37da48cfb2366b89f2c3c628" candidate_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5270a004873440ec9a5e8106b63a6a85"
I20260709 13:16:09.394222  8438 consensus_queue.cc:1059] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:16:09.388759 32224 raft_consensus.cc:2243] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.401134  8491 raft_consensus.cc:3055] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:09.401527  8491 raft_consensus.cc:740] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:09.402321  8491 consensus_queue.cc:260] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.389115  8460 consensus_queue.cc:1059] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:16:09.404425  8491 raft_consensus.cc:3060] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.404882  8460 raft_consensus.cc:3055] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:09.409587  8460 raft_consensus.cc:740] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:09.419203  8460 consensus_queue.cc:260] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } }
I20260709 13:16:09.420673  8460 raft_consensus.cc:3060] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:09.431541  7394 leader_election.cc:336] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:09.434384  7500 raft_consensus.cc:1240] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Rejecting Update request from peer 5270a004873440ec9a5e8106b63a6a85 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:09.432093  7394 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5270a004873440ec9a5e8106b63a6a85; no voters: 2b2b311085b64300925884dec9a28c7d, 49f7f0d3dec548bbb5538d2b1ddb4415
I20260709 13:16:09.435353  8438 raft_consensus.cc:2749] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.454478  7468 leader_election.cc:304] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.455276  8328 raft_consensus.cc:2749] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.408963  8067 leader_election.cc:304] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b2b311085b64300925884dec9a28c7d; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:09.459719  8446 raft_consensus.cc:2749] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.464740  8438 consensus_queue.cc:1059] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.465852  8448 raft_consensus.cc:3055] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:09.466202  8448 raft_consensus.cc:740] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5270a004873440ec9a5e8106b63a6a85, State: Running, Role: LEADER
I20260709 13:16:09.467020  8448 consensus_queue.cc:260] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:09.468356  8448 raft_consensus.cc:3060] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.468860  7179 catalog_manager.cc:5697] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 2 to 3, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.470487  7179 catalog_manager.cc:5697] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 reported cstate change: term changed from 1 to 2, leader changed from 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2) to 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "5270a004873440ec9a5e8106b63a6a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.472858 32224 raft_consensus.cc:2272] T 97a88676349042cda17ab04278fa8b6f P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:09.486423  8067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.3:35265, user_credentials={real_user=slave}} blocked reactor thread for 77531.2us
I20260709 13:16:09.515136 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.521854 32224 raft_consensus.cc:2243] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.530552 32224 raft_consensus.cc:2272] T 598d8f846ffc44fcaca77f1973eb6b7b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.532703  7422 raft_consensus.cc:1240] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Rejecting Update request from peer 2b2b311085b64300925884dec9a28c7d for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:09.540378 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.541661 32224 raft_consensus.cc:2243] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.542474 32224 raft_consensus.cc:2272] T be0584005e4e4271a39fc114468c717b P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.550909  8445 consensus_queue.cc:1059] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:09.552063  8445 raft_consensus.cc:3055] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:09.552397  8445 raft_consensus.cc:740] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2b2b311085b64300925884dec9a28c7d, State: Running, Role: LEADER
I20260709 13:16:09.553256  8445 consensus_queue.cc:260] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:09.554621  8445 raft_consensus.cc:3060] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.570886 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.571918 32224 raft_consensus.cc:2243] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.572496 32224 raft_consensus.cc:2272] T 623d8610e992488e9302c20596f4a302 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.575552 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.579934 32224 raft_consensus.cc:2243] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.580585 32224 raft_consensus.cc:2272] T 961fad27981341bb99577079ea5dcc3f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.584339 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.586014 32224 raft_consensus.cc:2243] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.587055 32224 raft_consensus.cc:2272] T ef5aa9b772e6445c83184f5551976222 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.590021 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.590852 32224 raft_consensus.cc:2243] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.591470 32224 raft_consensus.cc:2272] T 270e9deedc904538b2b42381a2aa3889 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.594936 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.595795 32224 raft_consensus.cc:2243] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.596871 32224 raft_consensus.cc:2272] T 0f21194252b0411c9c67f78fcfec3140 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.599752 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.600606 32224 raft_consensus.cc:2243] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.601116 32224 raft_consensus.cc:2272] T c92a5d8ff52e4a238b88b5dd6ed2317d P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.604219 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.605052 32224 raft_consensus.cc:2243] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.606238 32224 raft_consensus.cc:2272] T 3ad0292559534756b194428330c7c717 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.608928 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.609685 32224 raft_consensus.cc:2243] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.610647 32224 raft_consensus.cc:2272] T b6e534925d6a411990fea3bbc14a4935 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.614208 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.614993 32224 raft_consensus.cc:2243] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.615527 32224 raft_consensus.cc:2272] T f035dcc1e4cc4e6fae948203b417e7cd P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.618386 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.619168 32224 raft_consensus.cc:2243] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.619675 32224 raft_consensus.cc:2272] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.622403 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.623211 32224 raft_consensus.cc:2243] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.623764 32224 raft_consensus.cc:2272] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.627020 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.627818 32224 raft_consensus.cc:2243] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.628780 32224 raft_consensus.cc:2272] T 942a30e040ee4dcca90e52f6dc78a64d P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.631664 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.632491 32224 raft_consensus.cc:2243] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.633001 32224 raft_consensus.cc:2272] T ad9e725766854f9f8f7ab7741e09179b P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.635869 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.636611 32224 raft_consensus.cc:2243] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.637106 32224 raft_consensus.cc:2272] T 9d76f633b67148ef905cfa3b1d5c590e P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.639945 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.640650 32224 raft_consensus.cc:2243] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.641775 32224 raft_consensus.cc:2272] T 55b3b562b9fd453ab8b640d44359747a P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.644698 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.645248 32224 raft_consensus.cc:2243] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.645980 32224 raft_consensus.cc:2272] T 239bdb3893f6422a9456da1eec09ac9e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.647946 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.648543 32224 raft_consensus.cc:2243] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.648965 32224 raft_consensus.cc:2272] T f0ae5b6a026546828ed9671f9ca41aa0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.651216 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.652060 32224 raft_consensus.cc:2243] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.652918 32224 raft_consensus.cc:2272] T 1abce17e47154429b9f10b822f48ab3e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.655433 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.656131 32224 raft_consensus.cc:2243] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.657053 32224 raft_consensus.cc:2272] T 04afbc5d731346f9a3c99d9d89f097f0 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.659338 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.659884 32224 raft_consensus.cc:2243] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.660493 32224 pending_rounds.cc:70] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Trying to abort 1 pending ops.
I20260709 13:16:09.660730 32224 pending_rounds.cc:77] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305637761968623616 op_type: NO_OP noop_request { }
I20260709 13:16:09.661144 32224 raft_consensus.cc:2889] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:09.661446 32224 raft_consensus.cc:2272] T 141286c286284003b795291ad8766165 P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.664268 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.664927 32224 raft_consensus.cc:2243] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.665743 32224 raft_consensus.cc:2272] T fc3465e298174cd9b6cdf4a7ea730ebe P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.667843 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.668418 32224 raft_consensus.cc:2243] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.668833 32224 raft_consensus.cc:2272] T a71872601e484b06882011698f8b3b52 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.670668 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.671193 32224 raft_consensus.cc:2243] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.671669 32224 raft_consensus.cc:2272] T 56bbbbedcee0446abe898050f5dd1578 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.673889 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.674453 32224 raft_consensus.cc:2243] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.674922 32224 raft_consensus.cc:2272] T 2118345d07be4c3195f476caf906740a P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.677124 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.677641 32224 raft_consensus.cc:2243] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.678026 32224 raft_consensus.cc:2272] T 343f79626b484d4aa20e247aac64c837 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.680186 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.681062 32224 raft_consensus.cc:2243] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.681620 32224 raft_consensus.cc:2272] T 6e3fe3d05909404e9c7e64e0f20ba873 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.685799 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.686621 32224 raft_consensus.cc:2243] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.687678 32224 raft_consensus.cc:2272] T ce24f22b1cbe4955a186eb1a28aa32bf P 2b2b311085b64300925884dec9a28c7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.690316 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.691010 32224 raft_consensus.cc:2243] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.691556 32224 raft_consensus.cc:2272] T 8da8ac1115f84af29bb612331df632a6 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.694290 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.694957 32224 raft_consensus.cc:2243] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.695873 32224 raft_consensus.cc:2272] T 2aa90396c8ee443fa7e1f1e7308e594e P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.698490 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.699324 32224 raft_consensus.cc:2243] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.699903 32224 raft_consensus.cc:2272] T 2bd6aaff170d478f8d6920b5849c6978 P 2b2b311085b64300925884dec9a28c7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.702953 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:09.703642 32224 raft_consensus.cc:2243] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.704149 32224 raft_consensus.cc:2272] T 76381a75f921496eb6274bd7df95f601 P 2b2b311085b64300925884dec9a28c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.738215  7356 raft_consensus.cc:1275] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:09.739580  8449 consensus_queue.cc:1048] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:09.767891  7496 raft_consensus.cc:3060] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.776394  7496 raft_consensus.cc:1275] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:09.778314  8460 consensus_queue.cc:1048] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:09.820727 32224 tablet_server.cc:196] TabletServer@127.31.120.1:33107 shutdown complete.
I20260709 13:16:09.920490  7496 raft_consensus.cc:3060] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.930720 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:16:09.936036  7397 consensus_peers.cc:597] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.939342  7397 consensus_peers.cc:597] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.943528  7496 raft_consensus.cc:1275] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Refusing update from remote peer 5270a004873440ec9a5e8106b63a6a85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:09.946848  8460 consensus_queue.cc:1048] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:09.952737  7397 consensus_peers.cc:597] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.968632  7397 consensus_peers.cc:597] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.988696  7471 consensus_peers.cc:597] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:09.997910  7471 consensus_peers.cc:597] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.002297  7471 consensus_peers.cc:597] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.033713 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:10.035570 32224 tablet_replica.cc:333] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.036649 32224 raft_consensus.cc:2243] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.037791 32224 raft_consensus.cc:2272] T 8142e76af31447acb3186673c2a9934a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.038156  7397 consensus_peers.cc:597] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.040817 32224 tablet_replica.cc:333] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.041774 32224 raft_consensus.cc:2243] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.042779 32224 raft_consensus.cc:2272] T c86b5fcfd6d243e5940a37ab978b4f2b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.045339 32224 tablet_replica.cc:333] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.045904 32224 raft_consensus.cc:2243] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.046270 32224 raft_consensus.cc:2272] T aa22447b4d024658b6470d416092c4cc P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.048808 32224 tablet_replica.cc:333] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.049657 32224 raft_consensus.cc:2243] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.050191 32224 raft_consensus.cc:2272] T 8af02e3c522a4048b817ec444a3ad5e6 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.053036 32224 tablet_replica.cc:333] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.053753 32224 raft_consensus.cc:2243] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.054185 32224 raft_consensus.cc:2272] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.056248 32224 tablet_replica.cc:333] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.056808 32224 raft_consensus.cc:2243] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.057255 32224 raft_consensus.cc:2272] T ce24f22b1cbe4955a186eb1a28aa32bf P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.059320 32224 tablet_replica.cc:333] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.060012 32224 raft_consensus.cc:2243] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.060446 32224 raft_consensus.cc:2272] T aeedebd8085f46878c1a27393ea26f13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.062795 32224 tablet_replica.cc:333] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.063432 32224 raft_consensus.cc:2243] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.064432 32224 raft_consensus.cc:2272] T ad9e725766854f9f8f7ab7741e09179b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.067158 32224 tablet_replica.cc:333] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.068028 32224 raft_consensus.cc:2243] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.068547 32224 raft_consensus.cc:2272] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.071403 32224 tablet_replica.cc:333] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.072213 32224 raft_consensus.cc:2243] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.073419 32224 raft_consensus.cc:2272] T 9d76f633b67148ef905cfa3b1d5c590e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.076586 32224 tablet_replica.cc:333] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.077435 32224 raft_consensus.cc:2243] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.078007 32224 raft_consensus.cc:2272] T 27a84c07d08e47da9a71cd1c513c6105 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.080348 32224 tablet_replica.cc:333] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.081125 32224 raft_consensus.cc:2243] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.082005 32224 raft_consensus.cc:2272] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.084497 32224 tablet_replica.cc:333] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.085055 32224 raft_consensus.cc:2243] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.085623 32224 raft_consensus.cc:2272] T 90ec340d335041b0bc58e0bcdf0b8970 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.087833 32224 tablet_replica.cc:333] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.088397 32224 raft_consensus.cc:2243] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.089255 32224 raft_consensus.cc:2272] T 5a1a55fbf26d4f84bafaab065d4feece P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.091310 32224 tablet_replica.cc:333] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.091917 32224 raft_consensus.cc:2243] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.092986 32224 raft_consensus.cc:2272] T 9ec7093b8e1740cba58723ac7119486b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.095095 32224 tablet_replica.cc:333] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.095678 32224 raft_consensus.cc:2243] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.096475 32224 raft_consensus.cc:2272] T 961fad27981341bb99577079ea5dcc3f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.098738 32224 tablet_replica.cc:333] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.099334 32224 raft_consensus.cc:2243] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.099898 32224 raft_consensus.cc:2272] T c0c47eeb37da48cfb2366b89f2c3c628 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.102156 32224 tablet_replica.cc:333] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.102779 32224 raft_consensus.cc:2243] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.103268 32224 raft_consensus.cc:2272] T 623d8610e992488e9302c20596f4a302 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.105562 32224 tablet_replica.cc:333] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.106383 32224 raft_consensus.cc:2243] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.106905 32224 raft_consensus.cc:2272] T ef5aa9b772e6445c83184f5551976222 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.109261 32224 tablet_replica.cc:333] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.109959 32224 raft_consensus.cc:2243] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.110425 32224 raft_consensus.cc:2272] T 270e9deedc904538b2b42381a2aa3889 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.112591 32224 tablet_replica.cc:333] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.113327 32224 raft_consensus.cc:2243] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.113797 32224 raft_consensus.cc:2272] T 56bbbbedcee0446abe898050f5dd1578 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.116221 32224 tablet_replica.cc:333] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.116837 32224 raft_consensus.cc:2243] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.117375 32224 raft_consensus.cc:2272] T a71872601e484b06882011698f8b3b52 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.119560 32224 tablet_replica.cc:333] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.120262 32224 raft_consensus.cc:2243] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.120693 32224 raft_consensus.cc:2272] T d4eac7e41e3a46a2aed7ecb29109d99d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.122680 32224 tablet_replica.cc:333] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.123323 32224 raft_consensus.cc:2243] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.123812 32224 raft_consensus.cc:2272] T 2118345d07be4c3195f476caf906740a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.125828 32224 tablet_replica.cc:333] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.126332 32224 raft_consensus.cc:2243] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.126739 32224 raft_consensus.cc:2272] T 7cd43c082a974fbe87828b19abd5bc02 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.128957 32224 tablet_replica.cc:333] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.129678 32224 raft_consensus.cc:2243] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.130156 32224 raft_consensus.cc:2272] T be0584005e4e4271a39fc114468c717b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.132777 32224 tablet_replica.cc:333] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.133275 32224 raft_consensus.cc:2243] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.134114 32224 raft_consensus.cc:2272] T 0079a4cd3d134d079f559b4073824860 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.136307 32224 tablet_replica.cc:333] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.136859 32224 raft_consensus.cc:2243] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.137283 32224 raft_consensus.cc:2272] T 2aa90396c8ee443fa7e1f1e7308e594e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.139271 32224 tablet_replica.cc:333] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.139868 32224 raft_consensus.cc:2243] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.140316 32224 raft_consensus.cc:2272] T 2bd6aaff170d478f8d6920b5849c6978 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.142264 32224 tablet_replica.cc:333] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.142778 32224 raft_consensus.cc:2243] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.143565 32224 raft_consensus.cc:2272] T 76381a75f921496eb6274bd7df95f601 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.146006 32224 tablet_replica.cc:333] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.146705 32224 raft_consensus.cc:2243] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.147209 32224 raft_consensus.cc:2272] T fc3465e298174cd9b6cdf4a7ea730ebe P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.149813 32224 tablet_replica.cc:333] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.150584 32224 raft_consensus.cc:2243] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.151082 32224 raft_consensus.cc:2272] T 141286c286284003b795291ad8766165 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.153429 32224 tablet_replica.cc:333] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.153970 32224 raft_consensus.cc:2243] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.154397 32224 raft_consensus.cc:2272] T 239bdb3893f6422a9456da1eec09ac9e P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.156440 32224 tablet_replica.cc:333] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.156944 32224 raft_consensus.cc:2243] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.157394 32224 raft_consensus.cc:2272] T 271a5a5acaf443f6ae8e2122f6f16166 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.159416 32224 tablet_replica.cc:333] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.159950 32224 raft_consensus.cc:2243] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.160751 32224 raft_consensus.cc:2272] T f0ae5b6a026546828ed9671f9ca41aa0 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.163149 32224 tablet_replica.cc:333] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.163836 32224 raft_consensus.cc:2243] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.164311 32224 raft_consensus.cc:2272] T dc1db9e93f304836b7e9f3bc59c89600 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.166386 32224 tablet_replica.cc:333] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.166926 32224 raft_consensus.cc:2243] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.167346 32224 raft_consensus.cc:2272] T e1e566e4da5c4b478489086154cbbd13 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.170077 32224 tablet_replica.cc:333] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.170809 32224 raft_consensus.cc:2243] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.171260 32224 raft_consensus.cc:2272] T ad2b493283374c639936c29e00cdd96c P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.173516 32224 tablet_replica.cc:333] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.174049 32224 raft_consensus.cc:2243] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.174479 32224 raft_consensus.cc:2272] T 99f8dd60683a4a94b2eab776a941327a P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.176422 32224 tablet_replica.cc:333] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.176915 32224 raft_consensus.cc:2243] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.177345 32224 raft_consensus.cc:2272] T de79d0d3ecc64a7a96c61be7604e6753 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.179373 32224 tablet_replica.cc:333] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.179961 32224 raft_consensus.cc:2243] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.180799 32224 raft_consensus.cc:2272] T 598d8f846ffc44fcaca77f1973eb6b7b P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.182870 32224 tablet_replica.cc:333] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.183367 32224 raft_consensus.cc:2243] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.184367 32224 raft_consensus.cc:2272] T 46e82c6ed54a49e39710ec850e7ac826 P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.186777 32224 tablet_replica.cc:333] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
W20260709 13:16:10.187175  7397 consensus_peers.cc:597] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.187487 32224 raft_consensus.cc:2243] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.188179 32224 raft_consensus.cc:2272] T 97a88676349042cda17ab04278fa8b6f P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.190789 32224 tablet_replica.cc:333] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89: stopping tablet replica
I20260709 13:16:10.191362 32224 raft_consensus.cc:2243] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.191879 32224 raft_consensus.cc:2272] T c92a5d8ff52e4a238b88b5dd6ed2317d P 71e04cb9c24b4ddd9d56199a77f5aa89 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.199294  7397 consensus_peers.cc:597] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.205693  7471 consensus_peers.cc:597] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.223784  7471 consensus_peers.cc:597] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.229156  7397 consensus_peers.cc:597] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.262037  7471 consensus_peers.cc:597] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.273365 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
W20260709 13:16:10.281651  7397 consensus_peers.cc:597] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.282508  7397 consensus_peers.cc:597] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.285499  7397 consensus_peers.cc:597] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.311671  7397 consensus_peers.cc:597] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.368160 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
W20260709 13:16:10.403214  7471 consensus_peers.cc:597] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.420307  7397 consensus_peers.cc:597] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 -> Peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Couldn't send request to peer 71e04cb9c24b4ddd9d56199a77f5aa89. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.423331  7468 consensus_peers.cc:597] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.429051  7468 consensus_peers.cc:597] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.451828  7468 consensus_peers.cc:597] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.458448 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:10.459679 32224 tablet_replica.cc:333] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.460646 32224 raft_consensus.cc:2243] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.463317 32224 raft_consensus.cc:2272] T 8142e76af31447acb3186673c2a9934a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.466252 32224 tablet_replica.cc:333] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.466952 32224 raft_consensus.cc:2243] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.467517 32224 raft_consensus.cc:2272] T c86b5fcfd6d243e5940a37ab978b4f2b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.470088 32224 tablet_replica.cc:333] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.470957 32224 raft_consensus.cc:2243] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.472056 32224 raft_consensus.cc:2272] T aa22447b4d024658b6470d416092c4cc P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.476109 32224 tablet_replica.cc:333] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.477064 32224 raft_consensus.cc:2243] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.477537 32224 raft_consensus.cc:2272] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.480309 32224 tablet_replica.cc:333] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.481036 32224 raft_consensus.cc:2243] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.481534 32224 raft_consensus.cc:2272] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.484459 32224 tablet_replica.cc:333] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.485255 32224 raft_consensus.cc:2243] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.485752 32224 raft_consensus.cc:2272] T ce24f22b1cbe4955a186eb1a28aa32bf P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.488198 32224 tablet_replica.cc:333] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.488801 32224 raft_consensus.cc:2243] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.489293 32224 raft_consensus.cc:2272] T 8da8ac1115f84af29bb612331df632a6 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.492162 32224 tablet_replica.cc:333] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.493000 32224 raft_consensus.cc:2243] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.493939 32224 raft_consensus.cc:2272] T 6e3fe3d05909404e9c7e64e0f20ba873 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.494081  7468 consensus_peers.cc:597] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.496609 32224 tablet_replica.cc:333] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
W20260709 13:16:10.496915  7468 consensus_peers.cc:597] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.497509 32224 raft_consensus.cc:2243] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.497999 32224 raft_consensus.cc:2272] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.500221 32224 tablet_replica.cc:333] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.500931 32224 raft_consensus.cc:2243] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.501726 32224 raft_consensus.cc:2272] T 5e8365f37a3b4cdaa6fdfd3fb9f9e35f P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.503947 32224 tablet_replica.cc:333] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.504520 32224 raft_consensus.cc:2243] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.504972 32224 raft_consensus.cc:2272] T f035dcc1e4cc4e6fae948203b417e7cd P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.507213 32224 tablet_replica.cc:333] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.507864 32224 raft_consensus.cc:2243] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.508313 32224 raft_consensus.cc:2272] T 3ad0292559534756b194428330c7c717 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.510432 32224 tablet_replica.cc:333] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.511063 32224 raft_consensus.cc:2243] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.511514 32224 raft_consensus.cc:2272] T b6e534925d6a411990fea3bbc14a4935 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.513520 32224 tablet_replica.cc:333] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.514094 32224 raft_consensus.cc:2243] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.514506 32224 raft_consensus.cc:2272] T 90ec340d335041b0bc58e0bcdf0b8970 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.516587 32224 tablet_replica.cc:333] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.517150 32224 raft_consensus.cc:2243] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.518024 32224 raft_consensus.cc:2272] T 2bd6aaff170d478f8d6920b5849c6978 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.520187 32224 tablet_replica.cc:333] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.520748 32224 raft_consensus.cc:2243] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.521174 32224 raft_consensus.cc:2272] T 2aa90396c8ee443fa7e1f1e7308e594e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.523084 32224 tablet_replica.cc:333] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.523749 32224 raft_consensus.cc:2243] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.524199 32224 raft_consensus.cc:2272] T ad9e725766854f9f8f7ab7741e09179b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.526266 32224 tablet_replica.cc:333] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.527108 32224 raft_consensus.cc:2243] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.527572 32224 raft_consensus.cc:2272] T 942a30e040ee4dcca90e52f6dc78a64d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.530112 32224 tablet_replica.cc:333] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.530819 32224 raft_consensus.cc:2243] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.531312 32224 raft_consensus.cc:2272] T 271a5a5acaf443f6ae8e2122f6f16166 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.534041 32224 tablet_replica.cc:333] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.534791 32224 raft_consensus.cc:2243] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.535288 32224 raft_consensus.cc:2272] T f0ae5b6a026546828ed9671f9ca41aa0 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.537878 32224 tablet_replica.cc:333] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.538661 32224 raft_consensus.cc:2243] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.539158 32224 raft_consensus.cc:2272] T 56bce1894cd442d5bc2580cb8d02012d P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.541682 32224 tablet_replica.cc:333] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.542445 32224 raft_consensus.cc:2243] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.543026 32224 raft_consensus.cc:2272] T 1abce17e47154429b9f10b822f48ab3e P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.545727 32224 tablet_replica.cc:333] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.546561 32224 raft_consensus.cc:2243] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.547092 32224 raft_consensus.cc:2272] T 04afbc5d731346f9a3c99d9d89f097f0 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.549032 32224 tablet_replica.cc:333] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.549585 32224 raft_consensus.cc:2243] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.550012 32224 raft_consensus.cc:2272] T 0f21194252b0411c9c67f78fcfec3140 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.552302 32224 tablet_replica.cc:333] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.553017 32224 raft_consensus.cc:2243] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.553959 32224 raft_consensus.cc:2272] T 270e9deedc904538b2b42381a2aa3889 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.556339 32224 tablet_replica.cc:333] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.556905 32224 raft_consensus.cc:2243] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.558202 32224 raft_consensus.cc:2272] T 56bbbbedcee0446abe898050f5dd1578 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.561102 32224 tablet_replica.cc:333] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.561882 32224 raft_consensus.cc:2243] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.562975 32224 raft_consensus.cc:2272] T a71872601e484b06882011698f8b3b52 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.565470 32224 tablet_replica.cc:333] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.566182 32224 raft_consensus.cc:2243] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:16:10.566830  7468 consensus_peers.cc:597] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.567170 32224 raft_consensus.cc:2272] T d4eac7e41e3a46a2aed7ecb29109d99d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.569435 32224 tablet_replica.cc:333] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.570065 32224 raft_consensus.cc:2243] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.570482 32224 raft_consensus.cc:2272] T 7cd43c082a974fbe87828b19abd5bc02 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.573064 32224 tablet_replica.cc:333] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.573823 32224 raft_consensus.cc:2243] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.574811 32224 raft_consensus.cc:2272] T be0584005e4e4271a39fc114468c717b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.577320 32224 tablet_replica.cc:333] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.578125 32224 raft_consensus.cc:2243] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.578673 32224 raft_consensus.cc:2272] T 0079a4cd3d134d079f559b4073824860 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.581630 32224 tablet_replica.cc:333] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.582377 32224 raft_consensus.cc:2243] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.582852 32224 raft_consensus.cc:2272] T 343f79626b484d4aa20e247aac64c837 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.585330 32224 tablet_replica.cc:333] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.586102 32224 raft_consensus.cc:2243] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.586918 32224 raft_consensus.cc:2272] T e1e566e4da5c4b478489086154cbbd13 P 5270a004873440ec9a5e8106b63a6a85 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.589685 32224 tablet_replica.cc:333] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.590473 32224 raft_consensus.cc:2243] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.591455 32224 raft_consensus.cc:2272] T ad2b493283374c639936c29e00cdd96c P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.594045 32224 tablet_replica.cc:333] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.594769 32224 raft_consensus.cc:2243] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.595839 32224 raft_consensus.cc:2272] T 99f8dd60683a4a94b2eab776a941327a P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.598193 32224 tablet_replica.cc:333] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.598778 32224 raft_consensus.cc:2243] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.599264 32224 raft_consensus.cc:2272] T 178ebd1ec9704b868e8dec3ccdfad30d P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.601846 32224 tablet_replica.cc:333] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.602473 32224 raft_consensus.cc:2243] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.603395 32224 raft_consensus.cc:2272] T de79d0d3ecc64a7a96c61be7604e6753 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.605789 32224 tablet_replica.cc:333] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.606364 32224 raft_consensus.cc:2243] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.607271 32224 raft_consensus.cc:2272] T c92a5d8ff52e4a238b88b5dd6ed2317d P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.609438 32224 tablet_replica.cc:333] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.610044 32224 raft_consensus.cc:2243] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.610529 32224 raft_consensus.cc:2272] T 9ec7093b8e1740cba58723ac7119486b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.612685 32224 tablet_replica.cc:333] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.613246 32224 raft_consensus.cc:2243] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.613729 32224 raft_consensus.cc:2272] T c0c47eeb37da48cfb2366b89f2c3c628 P 5270a004873440ec9a5e8106b63a6a85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.615942 32224 tablet_replica.cc:333] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.616526 32224 raft_consensus.cc:2243] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.617019 32224 raft_consensus.cc:2272] T 9d76f633b67148ef905cfa3b1d5c590e P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.619248 32224 tablet_replica.cc:333] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.619870 32224 raft_consensus.cc:2243] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.620345 32224 raft_consensus.cc:2272] T 55b3b562b9fd453ab8b640d44359747a P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.622475 32224 tablet_replica.cc:333] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.622941 32224 raft_consensus.cc:2243] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.623370 32224 raft_consensus.cc:2272] T 46e82c6ed54a49e39710ec850e7ac826 P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.625710 32224 tablet_replica.cc:333] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85: stopping tablet replica
I20260709 13:16:10.626163 32224 raft_consensus.cc:2243] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.626662 32224 raft_consensus.cc:2272] T 9ad3d31ca56b45dda55d0616cc89204b P 5270a004873440ec9a5e8106b63a6a85 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.628674  7468 consensus_peers.cc:597] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.669550  8475 raft_consensus.cc:493] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:10.670207  8475 raft_consensus.cc:515] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:10.672641  8475 leader_election.cc:290] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
W20260709 13:16:10.677551  7468 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:10.678442  7468 leader_election.cc:336] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:16:10.678823  7468 leader_election.cc:304] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:10.679558  8475 raft_consensus.cc:2749] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:10.701308 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
W20260709 13:16:10.707216  7468 consensus_peers.cc:597] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Couldn't send request to peer 5270a004873440ec9a5e8106b63a6a85. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.3:35265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.730574  8475 raft_consensus.cc:493] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:10.731366  8475 raft_consensus.cc:515] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:10.734436  8475 leader_election.cc:290] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
W20260709 13:16:10.741086  7468 leader_election.cc:336] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Network error: Client connection negotiation failed: client connection to 127.31.120.3:35265: connect: Connection refused (error 111)
W20260709 13:16:10.741959  7468 leader_election.cc:336] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:16:10.742457  7468 leader_election.cc:304] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:10.743396  8475 raft_consensus.cc:2749] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:10.786523 32224 tablet_server.cc:179] TabletServer@127.31.120.4:0 shutting down...
I20260709 13:16:10.847311 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:10.848374 32224 tablet_replica.cc:333] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.849494 32224 raft_consensus.cc:2243] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.850260 32224 raft_consensus.cc:2272] T ef5aa9b772e6445c83184f5551976222 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.852831 32224 tablet_replica.cc:333] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.853499 32224 raft_consensus.cc:2243] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.853976 32224 raft_consensus.cc:2272] T 0f21194252b0411c9c67f78fcfec3140 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.856298 32224 tablet_replica.cc:333] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.856920 32224 raft_consensus.cc:2243] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.857412 32224 raft_consensus.cc:2272] T 8142e76af31447acb3186673c2a9934a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.859656 32224 tablet_replica.cc:333] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.860292 32224 raft_consensus.cc:2243] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.860777 32224 raft_consensus.cc:2272] T aa22447b4d024658b6470d416092c4cc P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.863389 32224 tablet_replica.cc:333] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.864074 32224 raft_consensus.cc:2243] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.865007 32224 raft_consensus.cc:2272] T 8af02e3c522a4048b817ec444a3ad5e6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.867272 32224 tablet_replica.cc:333] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.867974 32224 raft_consensus.cc:2243] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.868451 32224 raft_consensus.cc:2272] T 3b8f1e9ac0b246e5b46bd9d85336d5eb P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.870512 32224 tablet_replica.cc:333] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.871114 32224 raft_consensus.cc:2243] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.872100 32224 raft_consensus.cc:2272] T 2eb1ec31ba3340fb815cc11f5c116aa0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.874505 32224 tablet_replica.cc:333] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.875113 32224 raft_consensus.cc:2243] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.875988 32224 raft_consensus.cc:2272] T 8da8ac1115f84af29bb612331df632a6 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.878211 32224 tablet_replica.cc:333] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.878894 32224 raft_consensus.cc:2243] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.879472 32224 raft_consensus.cc:2272] T 6e3fe3d05909404e9c7e64e0f20ba873 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.880548  8521 raft_consensus.cc:493] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.881090  8521 raft_consensus.cc:515] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } } peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } }
I20260709 13:16:10.882579 32224 tablet_replica.cc:333] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.883576 32224 raft_consensus.cc:2243] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.884353  8521 leader_election.cc:290] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887), 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265)
I20260709 13:16:10.884922 32224 raft_consensus.cc:2272] T 27a84c07d08e47da9a71cd1c513c6105 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.887905 32224 tablet_replica.cc:333] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
W20260709 13:16:10.887960  7468 consensus_peers.cc:597] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 -> Peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Couldn't send request to peer 2b2b311085b64300925884dec9a28c7d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:10.888878 32224 raft_consensus.cc:2243] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.890095 32224 raft_consensus.cc:2272] T 178ebd1ec9704b868e8dec3ccdfad30d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.890000  7471 leader_election.cc:336] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111)
W20260709 13:16:10.891775  7468 leader_election.cc:336] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Network error: Client connection negotiation failed: client connection to 127.31.120.3:35265: connect: Connection refused (error 111)
I20260709 13:16:10.892232  7468 leader_election.cc:304] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 5270a004873440ec9a5e8106b63a6a85, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:10.892998  8521 raft_consensus.cc:2749] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:10.893594 32224 tablet_replica.cc:333] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.894354 32224 raft_consensus.cc:2243] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.894845 32224 raft_consensus.cc:2272] T 57da519e6ecb4abbaf3b5361a4e0b9ee P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.897671 32224 tablet_replica.cc:333] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.898576 32224 raft_consensus.cc:2243] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.898645  8521 raft_consensus.cc:493] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:10.899283 32224 raft_consensus.cc:2272] T 1abce17e47154429b9f10b822f48ab3e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.900005  8458 raft_consensus.cc:493] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:10.900509  8458 raft_consensus.cc:515] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:10.902390 32224 tablet_replica.cc:333] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.903348 32224 raft_consensus.cc:2243] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.903446  8475 raft_consensus.cc:493] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2b2b311085b64300925884dec9a28c7d)
I20260709 13:16:10.904165 32224 raft_consensus.cc:2272] T 04afbc5d731346f9a3c99d9d89f097f0 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.904122  8475 raft_consensus.cc:515] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } } peers { permanent_uuid: "71e04cb9c24b4ddd9d56199a77f5aa89" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35887 } }
I20260709 13:16:10.906791  8475 leader_election.cc:290] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:10.902894  8458 leader_election.cc:290] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107), 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887)
I20260709 13:16:10.907500 32224 tablet_replica.cc:333] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.908797 32224 raft_consensus.cc:2243] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.909567 32224 raft_consensus.cc:2272] T 9ec7093b8e1740cba58723ac7119486b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.899364  8521 raft_consensus.cc:515] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5270a004873440ec9a5e8106b63a6a85" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 35265 } } peers { permanent_uuid: "49f7f0d3dec548bbb5538d2b1ddb4415" member_type: VOTER last_known_addr { host: "127.31.120.4" port: 39007 } } peers { permanent_uuid: "2b2b311085b64300925884dec9a28c7d" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 33107 } }
I20260709 13:16:10.912060  8521 leader_election.cc:290] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265), 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107)
I20260709 13:16:10.913914 32224 tablet_replica.cc:333] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.915242 32224 raft_consensus.cc:2243] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.916080 32224 raft_consensus.cc:2272] T 961fad27981341bb99577079ea5dcc3f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.918608  7471 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111)
I20260709 13:16:10.919288 32224 tablet_replica.cc:333] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.920262 32224 raft_consensus.cc:2243] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.920862 32224 raft_consensus.cc:2272] T c0c47eeb37da48cfb2366b89f2c3c628 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.921453  7471 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 71e04cb9c24b4ddd9d56199a77f5aa89 (127.31.120.2:35887): Network error: Client connection negotiation failed: client connection to 127.31.120.2:35887: connect: Connection refused (error 111)
I20260709 13:16:10.923568 32224 tablet_replica.cc:333] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.924352 32224 raft_consensus.cc:2243] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.925384 32224 raft_consensus.cc:2272] T 623d8610e992488e9302c20596f4a302 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.928332 32224 tablet_replica.cc:333] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.929101 32224 raft_consensus.cc:2243] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:10.929170  7468 leader_election.cc:336] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:16:10.929692 32224 raft_consensus.cc:2272] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.929622  7468 leader_election.cc:304] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:10.930405  8521 raft_consensus.cc:2749] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:10.930555  7468 leader_election.cc:336] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:16:10.931084  7468 leader_election.cc:304] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 71e04cb9c24b4ddd9d56199a77f5aa89
I20260709 13:16:10.931861  8521 raft_consensus.cc:2749] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:10.932081  7468 leader_election.cc:336] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2b2b311085b64300925884dec9a28c7d (127.31.120.1:33107): Network error: Client connection negotiation failed: client connection to 127.31.120.1:33107: connect: Connection refused (error 111)
I20260709 13:16:10.932646 32224 tablet_replica.cc:333] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
W20260709 13:16:10.933219  7468 leader_election.cc:336] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5270a004873440ec9a5e8106b63a6a85 (127.31.120.3:35265): Network error: Client connection negotiation failed: client connection to 127.31.120.3:35265: connect: Connection refused (error 111)
I20260709 13:16:10.933465 32224 raft_consensus.cc:2243] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.933633  7468 leader_election.cc:304] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49f7f0d3dec548bbb5538d2b1ddb4415; no voters: 2b2b311085b64300925884dec9a28c7d, 5270a004873440ec9a5e8106b63a6a85
I20260709 13:16:10.934062 32224 raft_consensus.cc:2272] T 239bdb3893f6422a9456da1eec09ac9e P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.934250  7468 raft_consensus.cc:2686] T 942a30e040ee4dcca90e52f6dc78a64d P 49f7f0d3dec548bbb5538d2b1ddb4415: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:16:10.936714 32224 tablet_replica.cc:333] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.937520 32224 raft_consensus.cc:2243] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.938320 32224 raft_consensus.cc:2272] T 271a5a5acaf443f6ae8e2122f6f16166 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.940514 32224 tablet_replica.cc:333] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.941124 32224 raft_consensus.cc:2243] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.941577 32224 raft_consensus.cc:2272] T 5a1a55fbf26d4f84bafaab065d4feece P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.943572 32224 tablet_replica.cc:333] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.944197 32224 raft_consensus.cc:2243] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.945050 32224 raft_consensus.cc:2272] T f035dcc1e4cc4e6fae948203b417e7cd P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.947196 32224 tablet_replica.cc:333] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.947824 32224 raft_consensus.cc:2243] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.948616 32224 raft_consensus.cc:2272] T 3ad0292559534756b194428330c7c717 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.950675 32224 tablet_replica.cc:333] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.951222 32224 raft_consensus.cc:2243] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.952077 32224 raft_consensus.cc:2272] T 90ec340d335041b0bc58e0bcdf0b8970 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.954180 32224 tablet_replica.cc:333] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.954753 32224 raft_consensus.cc:2243] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.955215 32224 raft_consensus.cc:2272] T b6e534925d6a411990fea3bbc14a4935 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.957212 32224 tablet_replica.cc:333] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.957810 32224 raft_consensus.cc:2243] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.958680 32224 raft_consensus.cc:2272] T 55b3b562b9fd453ab8b640d44359747a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.960855 32224 tablet_replica.cc:333] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.961402 32224 raft_consensus.cc:2243] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.961877 32224 raft_consensus.cc:2272] T 97a88676349042cda17ab04278fa8b6f P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.963965 32224 tablet_replica.cc:333] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.964502 32224 raft_consensus.cc:2243] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.965276 32224 raft_consensus.cc:2272] T aeedebd8085f46878c1a27393ea26f13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.967422 32224 tablet_replica.cc:333] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.968278 32224 raft_consensus.cc:2243] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.969069 32224 raft_consensus.cc:2272] T 9ad3d31ca56b45dda55d0616cc89204b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.971122 32224 tablet_replica.cc:333] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.971761 32224 raft_consensus.cc:2243] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.972209 32224 raft_consensus.cc:2272] T d4eac7e41e3a46a2aed7ecb29109d99d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.974296 32224 tablet_replica.cc:333] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.974861 32224 raft_consensus.cc:2243] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.975704 32224 raft_consensus.cc:2272] T 2118345d07be4c3195f476caf906740a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.977706 32224 tablet_replica.cc:333] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.978232 32224 raft_consensus.cc:2243] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.978660 32224 raft_consensus.cc:2272] T 7cd43c082a974fbe87828b19abd5bc02 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.980708 32224 tablet_replica.cc:333] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.981256 32224 raft_consensus.cc:2243] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.981684 32224 raft_consensus.cc:2272] T 0079a4cd3d134d079f559b4073824860 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.983834 32224 tablet_replica.cc:333] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.984709 32224 raft_consensus.cc:2243] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.985278 32224 raft_consensus.cc:2272] T 99f8dd60683a4a94b2eab776a941327a P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.988270 32224 tablet_replica.cc:333] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.989086 32224 raft_consensus.cc:2243] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.989660 32224 raft_consensus.cc:2272] T ad2b493283374c639936c29e00cdd96c P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.992415 32224 tablet_replica.cc:333] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.993182 32224 raft_consensus.cc:2243] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.993742 32224 raft_consensus.cc:2272] T 598d8f846ffc44fcaca77f1973eb6b7b P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.995854 32224 tablet_replica.cc:333] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.996322 32224 raft_consensus.cc:2243] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.996680 32224 raft_consensus.cc:2272] T 343f79626b484d4aa20e247aac64c837 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.998545 32224 tablet_replica.cc:333] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:10.999068 32224 raft_consensus.cc:2243] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.999498 32224 raft_consensus.cc:2272] T dc1db9e93f304836b7e9f3bc59c89600 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.001611 32224 tablet_replica.cc:333] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:11.002125 32224 raft_consensus.cc:2243] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.002548 32224 raft_consensus.cc:2272] T e1e566e4da5c4b478489086154cbbd13 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.004463 32224 tablet_replica.cc:333] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:11.004959 32224 raft_consensus.cc:2243] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.005368 32224 raft_consensus.cc:2272] T 76381a75f921496eb6274bd7df95f601 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.007150 32224 tablet_replica.cc:333] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:11.007779 32224 raft_consensus.cc:2243] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.008225 32224 raft_consensus.cc:2272] T 56bce1894cd442d5bc2580cb8d02012d P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.010139 32224 tablet_replica.cc:333] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:11.010649 32224 raft_consensus.cc:2243] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.011070 32224 raft_consensus.cc:2272] T fc3465e298174cd9b6cdf4a7ea730ebe P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.012934 32224 tablet_replica.cc:333] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415: stopping tablet replica
I20260709 13:16:11.013401 32224 raft_consensus.cc:2243] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:11.013769 32224 raft_consensus.cc:2272] T 141286c286284003b795291ad8766165 P 49f7f0d3dec548bbb5538d2b1ddb4415 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.081785 32224 tablet_server.cc:196] TabletServer@127.31.120.4:0 shutdown complete.
I20260709 13:16:11.157465 32224 master.cc:562] Master@127.31.120.62:41611 shutting down...
I20260709 13:16:11.192792 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:11.193468 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:11.193892 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6aee7c93da9c4a9aacf1144b11e0129b: stopping tablet replica
W20260709 13:16:12.213866 32224 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 7211)
W20260709 13:16:13.214377 32224 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 7211)
I20260709 13:16:13.538456 32224 master.cc:584] Master@127.31.120.62:41611 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (36290 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:16:13.633628 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:41135
I20260709 13:16:13.634840 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:13.641120  8523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.641268  8524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.641487  8526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:13.642423 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:13.643577 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:13.643849 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:13.644032 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602973643998 us; error 0 us; skew 500 ppm
I20260709 13:16:13.644616 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:13.647058 32224 webserver.cc:533] Webserver started at http://127.31.120.62:37005/ using document root <none> and password file <none>
I20260709 13:16:13.647594 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:13.648109 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:13.648396 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:13.649484 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "732fdcdd3a714d76b8e1268e327c4b6f"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.654003 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:16:13.657891  8531 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:13.658778 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:13.659039 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "732fdcdd3a714d76b8e1268e327c4b6f"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.659281 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:13.684782 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:13.685986 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:13.726675 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:41135
I20260709 13:16:13.726795  8582 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:41135 every 8 connection(s)
I20260709 13:16:13.731099  8583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:13.744540  8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f: Bootstrap starting.
I20260709 13:16:13.749959  8583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.754747  8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f: No bootstrap required, opened a new log
I20260709 13:16:13.757202  8583 raft_consensus.cc:359] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER }
I20260709 13:16:13.757658  8583 raft_consensus.cc:385] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.757907  8583 raft_consensus.cc:740] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732fdcdd3a714d76b8e1268e327c4b6f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.758678  8583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER }
I20260709 13:16:13.759198  8583 raft_consensus.cc:399] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:13.759451  8583 raft_consensus.cc:493] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:13.759783  8583 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.765659  8583 raft_consensus.cc:515] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER }
I20260709 13:16:13.766279  8583 leader_election.cc:304] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 732fdcdd3a714d76b8e1268e327c4b6f; no voters: 
I20260709 13:16:13.767524  8583 leader_election.cc:290] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:13.767845  8586 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.769493  8586 raft_consensus.cc:697] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 1 LEADER]: Becoming Leader. State: Replica: 732fdcdd3a714d76b8e1268e327c4b6f, State: Running, Role: LEADER
I20260709 13:16:13.770202  8586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER }
I20260709 13:16:13.770994  8583 sys_catalog.cc:565] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:13.775198  8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 732fdcdd3a714d76b8e1268e327c4b6f. Latest consensus state: current_term: 1 leader_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER } }
I20260709 13:16:13.776082  8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:13.777119  8587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732fdcdd3a714d76b8e1268e327c4b6f" member_type: VOTER } }
I20260709 13:16:13.778616  8587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:13.779151  8593 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:13.784162  8593 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:13.788728 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:13.793467  8593 catalog_manager.cc:1383] Generated new cluster ID: bbf77f7c891b40d38b8d8abc47a73af3
I20260709 13:16:13.793812  8593 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:13.819216  8593 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:13.820735  8593 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:13.836221  8593 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f: Generated new TSK 0
I20260709 13:16:13.836889  8593 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:13.856842 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:13.863629  8604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.864142  8605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.866765  8607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:13.867285 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:13.868439 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:13.868621 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:13.868744 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602973868733 us; error 0 us; skew 500 ppm
I20260709 13:16:13.869230 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:13.871591 32224 webserver.cc:533] Webserver started at http://127.31.120.1:40043/ using document root <none> and password file <none>
I20260709 13:16:13.872102 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:13.872251 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:13.872525 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:13.873629 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.878017 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:13.881410  8612 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:13.882290 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:16:13.882627 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.882923 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:13.910482 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:13.911872 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:13.913329 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:13.915729 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:13.915947 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:13.916195 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:13.916385 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:13.958168 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:37209
I20260709 13:16:13.958420  8674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:37209 every 8 connection(s)
I20260709 13:16:13.963161 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:13.971107  8679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.972923  8680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:13.976367  8682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:13.976572 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:13.977838 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:13.978058 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:13.978255 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602973978237 us; error 0 us; skew 500 ppm
I20260709 13:16:13.978848 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:13.981809 32224 webserver.cc:533] Webserver started at http://127.31.120.2:43603/ using document root <none> and password file <none>
I20260709 13:16:13.982461 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:13.982724 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:13.983042 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:13.982921  8675 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41135
I20260709 13:16:13.983561  8675 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:13.984354  8675 heartbeater.cc:507] Master 127.31.120.62:41135 requested a full tablet report, sending...
I20260709 13:16:13.984413 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "eb5c1701174444b2ba83545192501b9c"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.986724  8548 ts_manager.cc:194] Registered new tserver with Master: 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:13.989235  8548 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59998
I20260709 13:16:13.990792 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.994328  8687 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:13.995152 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:16:13.995462 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "eb5c1701174444b2ba83545192501b9c"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:13.995801 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:14.009894 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:14.011137 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:14.012759 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:14.015044 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:14.015275 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:14.015532 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:14.015738 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:16:14.056492 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:36227
I20260709 13:16:14.056604  8749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:36227 every 8 connection(s)
I20260709 13:16:14.061335 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:14.068847  8753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:14.071350  8754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:14.072664  8756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:14.073189 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:14.074361 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:14.074648 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:14.074860 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602974074841 us; error 0 us; skew 500 ppm
I20260709 13:16:14.075285  8750 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41135
I20260709 13:16:14.075510 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:14.075856  8750 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:14.076903  8750 heartbeater.cc:507] Master 127.31.120.62:41135 requested a full tablet report, sending...
I20260709 13:16:14.078562 32224 webserver.cc:533] Webserver started at http://127.31.120.3:46559/ using document root <none> and password file <none>
I20260709 13:16:14.079284 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:14.079324  8548 ts_manager.cc:194] Registered new tserver with Master: eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:14.079650 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:14.080017 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:14.081337  8548 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60014
I20260709 13:16:14.081560 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "4c5e4b341ca14a52badd1585192b0cb3"
format_stamp: "Formatted at 2026-07-09 13:16:14 on dist-test-slave-pm98"
I20260709 13:16:14.086447 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:16:14.089668  8761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:14.090412 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:16:14.090667 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "4c5e4b341ca14a52badd1585192b0cb3"
format_stamp: "Formatted at 2026-07-09 13:16:14 on dist-test-slave-pm98"
I20260709 13:16:14.090895 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:14.102895 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:14.104121 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:14.105532 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:14.108421 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:14.108672 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:14.108914 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:14.109088 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:14.152554 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:33309
I20260709 13:16:14.152863  8823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:33309 every 8 connection(s)
I20260709 13:16:14.174106  8824 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41135
I20260709 13:16:14.174626  8824 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:14.175699  8824 heartbeater.cc:507] Master 127.31.120.62:41135 requested a full tablet report, sending...
I20260709 13:16:14.178349  8548 ts_manager.cc:194] Registered new tserver with Master: 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.178570 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019920108s
I20260709 13:16:14.180495  8548 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60020
I20260709 13:16:14.211035  8548 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60036:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:14.214260  8548 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:14.288976  8713 tablet_service.cc:1511] Processing CreateTablet for tablet aea636fd0e5648a883d808804e8ffc0f (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:14.290184  8715 tablet_service.cc:1511] Processing CreateTablet for tablet 78ecc05ba86a459c855f3ccd2e8f9d71 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:14.291160  8714 tablet_service.cc:1511] Processing CreateTablet for tablet b59fe8b5187f4b1e90cd8eabf6f74bec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:14.293041  8715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ecc05ba86a459c855f3ccd2e8f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.294291  8713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea636fd0e5648a883d808804e8ffc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.294691  8788 tablet_service.cc:1511] Processing CreateTablet for tablet b59fe8b5187f4b1e90cd8eabf6f74bec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:14.295251  8787 tablet_service.cc:1511] Processing CreateTablet for tablet aea636fd0e5648a883d808804e8ffc0f (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:14.296342  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59fe8b5187f4b1e90cd8eabf6f74bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.297132  8787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea636fd0e5648a883d808804e8ffc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.297804  8714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59fe8b5187f4b1e90cd8eabf6f74bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.297668  8789 tablet_service.cc:1511] Processing CreateTablet for tablet 78ecc05ba86a459c855f3ccd2e8f9d71 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:14.301502  8789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ecc05ba86a459c855f3ccd2e8f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.301261  8712 tablet_service.cc:1511] Processing CreateTablet for tablet b7a9971ce80048c182443a31df2193c2 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:14.303699  8712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a9971ce80048c182443a31df2193c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.309420  8711 tablet_service.cc:1511] Processing CreateTablet for tablet bd7e64c63b5d4981ab11de3b7b3b3a87 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:14.311092  8711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7e64c63b5d4981ab11de3b7b3b3a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.314743  8710 tablet_service.cc:1511] Processing CreateTablet for tablet 74246a9ea1cb44188bf02173c34162fc (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:14.316318  8710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74246a9ea1cb44188bf02173c34162fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.321805  8709 tablet_service.cc:1511] Processing CreateTablet for tablet 879a556aa631478fa439a825c5529391 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:14.323336  8709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a556aa631478fa439a825c5529391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.329470  8708 tablet_service.cc:1511] Processing CreateTablet for tablet 2aeaf1602891492a80c8c997936bc0ec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:14.329862  8786 tablet_service.cc:1511] Processing CreateTablet for tablet b7a9971ce80048c182443a31df2193c2 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:14.332397  8786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a9971ce80048c182443a31df2193c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.333154  8785 tablet_service.cc:1511] Processing CreateTablet for tablet bd7e64c63b5d4981ab11de3b7b3b3a87 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:14.334576  8785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7e64c63b5d4981ab11de3b7b3b3a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.340418  8784 tablet_service.cc:1511] Processing CreateTablet for tablet 74246a9ea1cb44188bf02173c34162fc (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:14.341984  8784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74246a9ea1cb44188bf02173c34162fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.344868  8783 tablet_service.cc:1511] Processing CreateTablet for tablet 879a556aa631478fa439a825c5529391 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:14.346342  8783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a556aa631478fa439a825c5529391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.351154  8708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aeaf1602891492a80c8c997936bc0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.356868  8707 tablet_service.cc:1511] Processing CreateTablet for tablet 04c0657dddef485fb8df5e83964123ef (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:14.358318  8707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c0657dddef485fb8df5e83964123ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.362460  8706 tablet_service.cc:1511] Processing CreateTablet for tablet 10393414464648d3840bc817aa4a70dd (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:14.364045  8706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10393414464648d3840bc817aa4a70dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.365571  8781 tablet_service.cc:1511] Processing CreateTablet for tablet 04c0657dddef485fb8df5e83964123ef (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:14.366977  8781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c0657dddef485fb8df5e83964123ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.371011  8782 tablet_service.cc:1511] Processing CreateTablet for tablet 2aeaf1602891492a80c8c997936bc0ec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:14.372503  8782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aeaf1602891492a80c8c997936bc0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.386489  8640 tablet_service.cc:1511] Processing CreateTablet for tablet 78ecc05ba86a459c855f3ccd2e8f9d71 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:14.388306  8632 tablet_service.cc:1511] Processing CreateTablet for tablet 04c0657dddef485fb8df5e83964123ef (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:14.388813  8638 tablet_service.cc:1511] Processing CreateTablet for tablet aea636fd0e5648a883d808804e8ffc0f (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:14.389844  8632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c0657dddef485fb8df5e83964123ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.391136  8636 tablet_service.cc:1511] Processing CreateTablet for tablet bd7e64c63b5d4981ab11de3b7b3b3a87 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:14.392685  8636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7e64c63b5d4981ab11de3b7b3b3a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.394171  8631 tablet_service.cc:1511] Processing CreateTablet for tablet 10393414464648d3840bc817aa4a70dd (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:14.394373  8635 tablet_service.cc:1511] Processing CreateTablet for tablet 74246a9ea1cb44188bf02173c34162fc (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:14.395674  8631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10393414464648d3840bc817aa4a70dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.396832  8633 tablet_service.cc:1511] Processing CreateTablet for tablet 2aeaf1602891492a80c8c997936bc0ec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:14.398998  8639 tablet_service.cc:1511] Processing CreateTablet for tablet b59fe8b5187f4b1e90cd8eabf6f74bec (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:14.401080  8637 tablet_service.cc:1511] Processing CreateTablet for tablet b7a9971ce80048c182443a31df2193c2 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:14.404280  8634 tablet_service.cc:1511] Processing CreateTablet for tablet 879a556aa631478fa439a825c5529391 (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:14.405998  8633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aeaf1602891492a80c8c997936bc0ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.408272  8639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59fe8b5187f4b1e90cd8eabf6f74bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.412294  8637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a9971ce80048c182443a31df2193c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.416246  8634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a556aa631478fa439a825c5529391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.423790  8635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74246a9ea1cb44188bf02173c34162fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.433921  8640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78ecc05ba86a459c855f3ccd2e8f9d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.435984  8638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea636fd0e5648a883d808804e8ffc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.447582  8839 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.450425  8789 tablet_service.cc:1511] Processing CreateTablet for tablet 10393414464648d3840bc817aa4a70dd (DEFAULT_TABLE table=test-workload [id=48b203b24c3944d281646904526272d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:14.458714  8789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10393414464648d3840bc817aa4a70dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:14.471022  8840 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.475970  8839 tablet_bootstrap.cc:654] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.486358  8841 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:14.521317  8841 tablet_bootstrap.cc:654] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.531208  8840 tablet_bootstrap.cc:654] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.534233  8839 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.545987  8839 ts_tablet_manager.cc:1403] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.019s
I20260709 13:16:14.550398  8841 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:14.551014  8841 ts_tablet_manager.cc:1403] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.001s
I20260709 13:16:14.551836  8839 raft_consensus.cc:359] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.552832  8839 raft_consensus.cc:385] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.553133  8839 raft_consensus.cc:740] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.553994  8840 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.553876  8839 consensus_queue.cc:260] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.553884  8841 raft_consensus.cc:359] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.554701  8840 ts_tablet_manager.cc:1403] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.000s
I20260709 13:16:14.554926  8841 raft_consensus.cc:385] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.555289  8841 raft_consensus.cc:740] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.556154  8841 consensus_queue.cc:260] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.556936  8839 ts_tablet_manager.cc:1434] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:16:14.558012  8839 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.558001  8840 raft_consensus.cc:359] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.558764  8841 ts_tablet_manager.cc:1434] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:16:14.558889  8840 raft_consensus.cc:385] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.559303  8840 raft_consensus.cc:740] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.559896  8841 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:14.609503  8675 heartbeater.cc:499] Master 127.31.120.62:41135 was elected leader, sending a full tablet report...
W20260709 13:16:14.618781  8751 tablet.cc:2406] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:14.624796  8750 heartbeater.cc:499] Master 127.31.120.62:41135 was elected leader, sending a full tablet report...
I20260709 13:16:14.560083  8840 consensus_queue.cc:260] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.647495  8841 tablet_bootstrap.cc:654] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.653347  8824 heartbeater.cc:499] Master 127.31.120.62:41135 was elected leader, sending a full tablet report...
I20260709 13:16:14.657583  8840 ts_tablet_manager.cc:1434] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.102s	user 0.011s	sys 0.000s
I20260709 13:16:14.659260  8840 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.672942  8841 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:14.661839  8839 tablet_bootstrap.cc:654] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.674991  8840 tablet_bootstrap.cc:654] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.675033  8841 ts_tablet_manager.cc:1403] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.115s	user 0.017s	sys 0.010s
I20260709 13:16:14.680322  8841 raft_consensus.cc:359] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.681998  8841 raft_consensus.cc:385] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.682459  8841 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.683732  8841 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.686455  8839 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.687143  8841 ts_tablet_manager.cc:1434] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:16:14.687252  8839 ts_tablet_manager.cc:1403] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.129s	user 0.020s	sys 0.004s
I20260709 13:16:14.688768  8841 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:14.691262  8839 raft_consensus.cc:359] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.693096  8839 raft_consensus.cc:385] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.693552  8839 raft_consensus.cc:740] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.695248  8839 consensus_queue.cc:260] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.696120  8840 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.696764  8840 ts_tablet_manager.cc:1403] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.019s
I20260709 13:16:14.696929  8847 raft_consensus.cc:493] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.697631  8847 raft_consensus.cc:515] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.698550  8839 ts_tablet_manager.cc:1434] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:16:14.699975  8839 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.700555  8840 raft_consensus.cc:359] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.701339  8840 raft_consensus.cc:385] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.701694  8840 raft_consensus.cc:740] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.702867  8840 consensus_queue.cc:260] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.705585  8840 ts_tablet_manager.cc:1434] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:14.706867  8840 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.707965  8847 leader_election.cc:290] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.714917  8841 tablet_bootstrap.cc:654] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.717397  8840 tablet_bootstrap.cc:654] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.718317  8839 tablet_bootstrap.cc:654] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.731416  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:14.731549  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:14.732765  8841 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:14.733323  8841 ts_tablet_manager.cc:1403] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.045s	user 0.005s	sys 0.008s
W20260709 13:16:14.733424  8613 leader_election.cc:343] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:14.734195  8614 leader_election.cc:343] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.734793  8614 leader_election.cc:304] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8; no voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c
I20260709 13:16:14.735460  8847 raft_consensus.cc:2749] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.737524  8840 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.737080  8841 raft_consensus.cc:359] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.737919  8841 raft_consensus.cc:385] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.738060  8840 ts_tablet_manager.cc:1403] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.008s
I20260709 13:16:14.738229  8841 raft_consensus.cc:740] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.738539  8839 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.739070  8839 ts_tablet_manager.cc:1403] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 13:16:14.739207  8841 consensus_queue.cc:260] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.741483  8840 raft_consensus.cc:359] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.742241  8840 raft_consensus.cc:385] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.742137  8839 raft_consensus.cc:359] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.742659  8840 raft_consensus.cc:740] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.742960  8839 raft_consensus.cc:385] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.743281  8839 raft_consensus.cc:740] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.743580  8840 consensus_queue.cc:260] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.744099  8839 consensus_queue.cc:260] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.745679  8848 raft_consensus.cc:493] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.746104  8848 raft_consensus.cc:515] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.746654  8839 ts_tablet_manager.cc:1434] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:16:14.747936  8839 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.749081  8848 leader_election.cc:290] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:14.755194  8839 tablet_bootstrap.cc:654] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.756541  8840 ts_tablet_manager.cc:1434] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:16:14.758085  8840 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.767464  8841 ts_tablet_manager.cc:1434] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:16:14.767745  8840 tablet_bootstrap.cc:654] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.774358  8847 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.775015  8847 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.780808  8847 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.780690  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:14.781786  8839 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.782105  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:14.782435  8839 ts_tablet_manager.cc:1403] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.008s
I20260709 13:16:14.781672  8725 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 0.
I20260709 13:16:14.777206  8841 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:14.785452  8613 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
W20260709 13:16:14.787074  8614 leader_election.cc:343] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:14.791452  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:14.792291  8847 raft_consensus.cc:2804] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.792760  8847 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.793109  8847 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.793785  8841 tablet_bootstrap.cc:654] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.797676  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
W20260709 13:16:14.800417  8762 leader_election.cc:343] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.803260  8839 raft_consensus.cc:359] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.805910  8841 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:14.810690  8841 ts_tablet_manager.cc:1403] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.009s
I20260709 13:16:14.809031  8847 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.809947  8840 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.813530  8840 ts_tablet_manager.cc:1403] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.006s
I20260709 13:16:14.814332  8839 raft_consensus.cc:385] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.814214  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:14.814834  8839 raft_consensus.cc:740] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.814230  8841 raft_consensus.cc:359] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.815475  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:14.815995  8841 raft_consensus.cc:385] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.816179  8725 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.815821  8839 consensus_queue.cc:260] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.816394  8841 raft_consensus.cc:740] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.816991  8840 raft_consensus.cc:359] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.817821  8840 raft_consensus.cc:385] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.817487  8841 consensus_queue.cc:260] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.818130  8840 raft_consensus.cc:740] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.818531  8650 raft_consensus.cc:2468] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 0.
I20260709 13:16:14.819242  8840 consensus_queue.cc:260] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.822341  8763 leader_election.cc:304] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: eb5c1701174444b2ba83545192501b9c
I20260709 13:16:14.828562  8857 raft_consensus.cc:2804] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.829164  8857 raft_consensus.cc:493] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.829627  8857 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.832998  8840 ts_tablet_manager.cc:1434] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.001s
I20260709 13:16:14.832995  8839 ts_tablet_manager.cc:1434] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.003s
I20260709 13:16:14.835196  8840 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.837152  8839 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.846601  8840 tablet_bootstrap.cc:654] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.850893  8839 tablet_bootstrap.cc:654] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.852916  8725 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 1.
I20260709 13:16:14.856081  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
W20260709 13:16:14.858184  8614 leader_election.cc:343] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.858994  8847 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Requested vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.861192  8840 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.862067  8840 ts_tablet_manager.cc:1403] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.013s
I20260709 13:16:14.863022  8839 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.863592  8839 ts_tablet_manager.cc:1403] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.008s
I20260709 13:16:14.867085  8840 raft_consensus.cc:359] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.867537  8839 raft_consensus.cc:359] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.868496  8839 raft_consensus.cc:385] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.868527  8840 raft_consensus.cc:385] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.868935  8839 raft_consensus.cc:740] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.869463  8840 raft_consensus.cc:740] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.870139  8839 consensus_queue.cc:260] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.871341  8613 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:14.870954  8840 consensus_queue.cc:260] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.872633  8847 raft_consensus.cc:2804] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.873144  8847 raft_consensus.cc:697] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Running, Role: LEADER
I20260709 13:16:14.874078  8847 consensus_queue.cc:237] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.874868  8840 ts_tablet_manager.cc:1434] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:16:14.876051  8857 raft_consensus.cc:515] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.879433  8840 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:14.883991  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
W20260709 13:16:14.886332  8762 leader_election.cc:343] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.888756  8857 leader_election.cc:290] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Requested vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:14.889007  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:14.889729  8650 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.891340  8839 ts_tablet_manager.cc:1434] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.005s
I20260709 13:16:14.902889  8839 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:14.903808  8841 ts_tablet_manager.cc:1434] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.093s	user 0.028s	sys 0.047s
I20260709 13:16:14.905185  8841 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:14.909746  8650 raft_consensus.cc:2468] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:14.910982  8763 leader_election.cc:304] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: eb5c1701174444b2ba83545192501b9c
I20260709 13:16:14.910868  8546 catalog_manager.cc:5697] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.913414  8857 raft_consensus.cc:2804] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.913968  8857 raft_consensus.cc:697] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:14.914912  8857 consensus_queue.cc:237] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.916633  8848 raft_consensus.cc:493] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.917162  8848 raft_consensus.cc:515] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.920727  8839 tablet_bootstrap.cc:654] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.921075  8840 tablet_bootstrap.cc:654] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.920801  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
W20260709 13:16:14.922891  8763 leader_election.cc:343] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.924268  8848 leader_election.cc:290] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:14.925011  8848 raft_consensus.cc:493] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.926635  8848 raft_consensus.cc:515] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.930119  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:14.931942  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
W20260709 13:16:14.934384  8763 leader_election.cc:343] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:14.935746  8762 leader_election.cc:343] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.937199  8762 leader_election.cc:304] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c
I20260709 13:16:14.941612  8841 tablet_bootstrap.cc:654] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.943933  8846 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.944711  8846 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.946735  8869 raft_consensus.cc:2749] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.950416  8846 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.954324  8846 raft_consensus.cc:493] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.972431  8839 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:14.971894  8846 raft_consensus.cc:515] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.971762  8548 catalog_manager.cc:5697] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.972993  8839 ts_tablet_manager.cc:1403] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.018s
I20260709 13:16:14.974926  8846 leader_election.cc:290] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:14.976316  8839 raft_consensus.cc:359] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:14.976341  8848 leader_election.cc:290] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:14.978396  8839 raft_consensus.cc:385] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.979171  8839 raft_consensus.cc:740] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.979234  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:14.980453  8839 consensus_queue.cc:260] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
W20260709 13:16:14.981010  8762 leader_election.cc:343] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.981477  8762 leader_election.cc:304] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c
I20260709 13:16:14.982590  8848 raft_consensus.cc:2749] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.988266  8840 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:14.991446  8840 ts_tablet_manager.cc:1403] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.112s	user 0.022s	sys 0.006s
I20260709 13:16:14.995054  8840 raft_consensus.cc:359] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.995842  8840 raft_consensus.cc:385] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.996110  8840 raft_consensus.cc:740] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.996371  8841 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:14.990621  8839 ts_tablet_manager.cc:1434] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:16:14.996942  8841 ts_tablet_manager.cc:1403] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.004s
I20260709 13:16:14.998159  8839 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:15.000527  8841 raft_consensus.cc:359] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.001295  8841 raft_consensus.cc:385] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.001582  8841 raft_consensus.cc:740] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.002354  8841 consensus_queue.cc:260] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:14.996963  8840 consensus_queue.cc:260] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.004997  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.006558  8840 ts_tablet_manager.cc:1434] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:16:15.007266  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.007828  8840 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:15.007999  8799 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
I20260709 13:16:15.010392  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:15.012550  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:15.013206  8650 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
W20260709 13:16:15.014690  8689 leader_election.cc:343] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.015305  8839 tablet_bootstrap.cc:654] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.015424  8689 leader_election.cc:304] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.015947  8840 tablet_bootstrap.cc:654] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.023540  8846 raft_consensus.cc:2804] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.023783  8839 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:15.024031  8846 raft_consensus.cc:493] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.024442  8846 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.024463  8839 ts_tablet_manager.cc:1403] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 13:16:15.027987  8839 raft_consensus.cc:359] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.028689  8839 raft_consensus.cc:385] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.028976  8839 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.029891  8839 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.032168  8839 ts_tablet_manager.cc:1434] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:15.016872  8841 ts_tablet_manager.cc:1434] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:16:15.033515  8839 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:15.033277  8846 raft_consensus.cc:515] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
W20260709 13:16:15.035930  8689 leader_election.cc:343] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.036429  8689 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:15.037408  8872 raft_consensus.cc:2749] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.035528  8846 leader_election.cc:290] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.039587  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.040016  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.040262  8650 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.040613  8799 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.041352  8839 tablet_bootstrap.cc:654] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.043561  8840 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:15.049417  8840 ts_tablet_manager.cc:1403] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.004s
I20260709 13:16:15.048044  8650 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.053728  8799 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.053902  8689 leader_election.cc:304] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.054080  8840 raft_consensus.cc:359] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.055158  8840 raft_consensus.cc:385] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.055409  8841 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:15.055523  8840 raft_consensus.cc:740] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.055778  8846 raft_consensus.cc:2804] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.055972  8839 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:15.057508  8839 ts_tablet_manager.cc:1403] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.002s
I20260709 13:16:15.057525  8846 raft_consensus.cc:697] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:15.057639  8840 consensus_queue.cc:260] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.058699  8846 consensus_queue.cc:237] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.060391  8840 ts_tablet_manager.cc:1434] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:16:15.061300  8839 raft_consensus.cc:359] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.061898  8840 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:15.062037  8839 raft_consensus.cc:385] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.062505  8839 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.063514  8839 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.065901  8839 ts_tablet_manager.cc:1434] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:15.067018  8839 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:15.070323  8840 tablet_bootstrap.cc:654] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.073752  8839 tablet_bootstrap.cc:654] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.082666  8840 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:15.083192  8840 ts_tablet_manager.cc:1403] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.011s
I20260709 13:16:15.084069  8841 tablet_bootstrap.cc:654] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.084443  8839 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:15.084940  8839 ts_tablet_manager.cc:1403] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:16:15.086071  8840 raft_consensus.cc:359] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.086844  8840 raft_consensus.cc:385] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.087113  8840 raft_consensus.cc:740] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.087757  8839 raft_consensus.cc:359] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.087893  8840 consensus_queue.cc:260] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.088524  8839 raft_consensus.cc:385] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.088874  8839 raft_consensus.cc:740] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.089754  8839 consensus_queue.cc:260] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.090186  8840 ts_tablet_manager.cc:1434] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:15.091275  8840 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:15.092196  8839 ts_tablet_manager.cc:1434] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:15.092644  8847 raft_consensus.cc:493] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.093261  8839 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: Bootstrap starting.
I20260709 13:16:15.093349  8847 raft_consensus.cc:515] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.097841  8840 tablet_bootstrap.cc:654] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.097793  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.098915  8725 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 0.
I20260709 13:16:15.100651  8839 tablet_bootstrap.cc:654] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.101423  8613 leader_election.cc:304] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.102885  8863 raft_consensus.cc:2804] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.103418  8863 raft_consensus.cc:493] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.103902  8863 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.106089  8846 raft_consensus.cc:493] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.106828  8846 raft_consensus.cc:515] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:15.109958  8846 leader_election.cc:290] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309), 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:15.111706  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:15.112605  8650 raft_consensus.cc:2393] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb5c1701174444b2ba83545192501b9c in current term 1: Already voted for candidate 4c5e4b341ca14a52badd1585192b0cb3 in this term.
I20260709 13:16:15.120229  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.122143  8689 leader_election.cc:304] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:15.123107  8847 leader_election.cc:290] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.123370  8846 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.087399  8547 catalog_manager.cc:5697] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 1 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.130925  8846 raft_consensus.cc:2749] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.141426  8846 raft_consensus.cc:493] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.141942  8846 raft_consensus.cc:515] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.144192  8846 leader_election.cc:290] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.146049  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
W20260709 13:16:15.149636  8689 leader_election.cc:343] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.147584  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.150578  8798 raft_consensus.cc:2468] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
I20260709 13:16:15.152261  8689 leader_election.cc:304] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:15.153357  8872 raft_consensus.cc:493] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.153810  8872 raft_consensus.cc:515] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.156174  8872 leader_election.cc:290] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.156736  8872 raft_consensus.cc:2804] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.157033  8872 raft_consensus.cc:493] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.157361  8872 raft_consensus.cc:3060] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.156023  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.158453  8799 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 0.
I20260709 13:16:15.161692  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
W20260709 13:16:15.162994  8689 leader_election.cc:343] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.161692  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
W20260709 13:16:15.167830  8689 leader_election.cc:343] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.168376  8689 leader_election.cc:304] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:15.169627  8846 raft_consensus.cc:2749] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.168216  8863 raft_consensus.cc:515] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.172600  8840 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:15.173137  8840 ts_tablet_manager.cc:1403] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.082s	user 0.030s	sys 0.049s
I20260709 13:16:15.173363  8839 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: No bootstrap required, opened a new log
I20260709 13:16:15.173288  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:15.174088  8839 ts_tablet_manager.cc:1403] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: Time spent bootstrapping tablet: real 0.081s	user 0.037s	sys 0.035s
I20260709 13:16:15.174185  8725 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.175364  8872 raft_consensus.cc:515] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.176043  8840 raft_consensus.cc:359] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.176707  8840 raft_consensus.cc:385] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.177017  8840 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.176843  8839 raft_consensus.cc:359] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.177536  8872 leader_election.cc:290] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.177569  8839 raft_consensus.cc:385] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.173249  8841 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:15.178143  8839 raft_consensus.cc:740] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.177848  8840 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.178643  8841 ts_tablet_manager.cc:1403] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.027s
I20260709 13:16:15.179116  8839 consensus_queue.cc:260] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.180994  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.181972  8841 raft_consensus.cc:359] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.182734  8841 raft_consensus.cc:385] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.183038  8841 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.184291  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.184999  8799 raft_consensus.cc:3060] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.187194  8840 ts_tablet_manager.cc:1434] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:16:15.188401  8840 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: Bootstrap starting.
I20260709 13:16:15.188869  8863 leader_election.cc:290] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Requested vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.190605  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.191269  8798 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.181332  8839 ts_tablet_manager.cc:1434] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 13:16:15.185030  8689 leader_election.cc:343] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.194909  8799 raft_consensus.cc:2468] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.195297  8840 tablet_bootstrap.cc:654] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.195669  8841 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.198365  8841 ts_tablet_manager.cc:1434] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 13:16:15.198987  8689 leader_election.cc:304] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:15.201416  8841 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:15.202069  8846 raft_consensus.cc:2804] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.189855  8725 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 1.
I20260709 13:16:15.209255  8848 raft_consensus.cc:493] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.209964  8841 tablet_bootstrap.cc:654] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.207515  8798 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 1.
I20260709 13:16:15.210846  8613 leader_election.cc:304] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.209955  8848 raft_consensus.cc:515] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.214869  8846 raft_consensus.cc:697] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:15.216113  8841 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:15.216001  8848 leader_election.cc:290] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:15.216881  8841 ts_tablet_manager.cc:1403] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 13:16:15.215902  8846 consensus_queue.cc:237] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.218160  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879a556aa631478fa439a825c5529391" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
W20260709 13:16:15.219991  8763 leader_election.cc:343] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.220188  8841 raft_consensus.cc:359] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.221010  8841 raft_consensus.cc:385] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.221071  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879a556aa631478fa439a825c5529391" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.221402  8841 raft_consensus.cc:740] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.221784  8725 raft_consensus.cc:2468] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 0.
I20260709 13:16:15.222246  8841 consensus_queue.cc:260] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.223183  8762 leader_election.cc:304] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:15.223328  8889 raft_consensus.cc:493] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.223433  8847 raft_consensus.cc:2804] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.224627  8848 raft_consensus.cc:2804] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.225049  8848 raft_consensus.cc:493] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.225442  8848 raft_consensus.cc:3060] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.225749  8841 ts_tablet_manager.cc:1434] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:16:15.226930  8841 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:15.225185  8889 raft_consensus.cc:515] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.233404  8848 raft_consensus.cc:515] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.234169  8841 tablet_bootstrap.cc:654] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.234611  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
W20260709 13:16:15.237139  8689 leader_election.cc:343] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.237534  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879a556aa631478fa439a825c5529391" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.238111  8650 raft_consensus.cc:3060] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.241695  8848 leader_election.cc:290] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:15.242808  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879a556aa631478fa439a825c5529391" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:15.243917  8725 raft_consensus.cc:3060] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.246280  8650 raft_consensus.cc:2468] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:15.242544  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.247696  8763 leader_election.cc:304] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:15.248129  8798 raft_consensus.cc:2468] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
I20260709 13:16:15.242882  8889 leader_election.cc:290] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.249653  8689 leader_election.cc:304] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:15.249277  8840 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: No bootstrap required, opened a new log
I20260709 13:16:15.250099  8847 raft_consensus.cc:697] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Running, Role: LEADER
I20260709 13:16:15.251420  8872 raft_consensus.cc:2804] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.253168  8872 raft_consensus.cc:493] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.253507  8872 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.255795  8848 raft_consensus.cc:2804] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.252133  8840 ts_tablet_manager.cc:1403] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.012s
I20260709 13:16:15.257849  8841 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:15.258328  8841 ts_tablet_manager.cc:1403] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.007s
I20260709 13:16:15.259347  8840 raft_consensus.cc:359] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.259771  8547 catalog_manager.cc:5697] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 1 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.256383  8848 raft_consensus.cc:697] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:15.261400  8841 raft_consensus.cc:359] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.252705  8847 consensus_queue.cc:237] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.262369  8841 raft_consensus.cc:385] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.260110  8840 raft_consensus.cc:385] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.262939  8840 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.262665  8848 consensus_queue.cc:237] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.260129  8725 raft_consensus.cc:2468] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:15.262712  8841 raft_consensus.cc:740] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.265519  8840 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.266801  8841 consensus_queue.cc:260] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.267617  8840 ts_tablet_manager.cc:1434] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:16:15.269882  8841 ts_tablet_manager.cc:1434] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:16:15.271253  8841 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:15.272135  8872 raft_consensus.cc:515] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.275413  8872 leader_election.cc:290] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.276829  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.278513  8841 tablet_bootstrap.cc:654] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:15.279235  8689 leader_election.cc:343] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.281546  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.282212  8798 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.285570  8548 catalog_manager.cc:5697] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.285426  8547 catalog_manager.cc:5697] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.290242  8798 raft_consensus.cc:2468] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.291879  8689 leader_election.cc:304] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:15.292919  8872 raft_consensus.cc:2804] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.293462  8872 raft_consensus.cc:697] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:15.293937  8841 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:15.294577  8841 ts_tablet_manager.cc:1403] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:16:15.294301  8872 consensus_queue.cc:237] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.295756  8889 raft_consensus.cc:493] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.296271  8889 raft_consensus.cc:515] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.297680  8841 raft_consensus.cc:359] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.298309  8889 leader_election.cc:290] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.298448  8841 raft_consensus.cc:385] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.298933  8841 raft_consensus.cc:740] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.299680  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:15.300345  8650 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
I20260709 13:16:15.299837  8841 consensus_queue.cc:260] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.300715  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.301414  8798 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 0.
I20260709 13:16:15.302093  8689 leader_election.cc:304] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.298170  8847 consensus_queue.cc:1048] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:15.303673  8841 ts_tablet_manager.cc:1434] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:15.305048  8841 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:15.306613  8889 raft_consensus.cc:2804] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.307039  8889 raft_consensus.cc:493] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.307374  8889 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.313365  8841 tablet_bootstrap.cc:654] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.326025  8798 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.329097  8848 consensus_queue.cc:1048] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:16:15.330551  8545 catalog_manager.cc:5697] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 1 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.341884  8847 consensus_queue.cc:1048] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.344301  8841 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: No bootstrap required, opened a new log
I20260709 13:16:15.345027  8841 ts_tablet_manager.cc:1403] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.014s
I20260709 13:16:15.348905  8841 raft_consensus.cc:359] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.349817  8841 raft_consensus.cc:385] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.350178  8841 raft_consensus.cc:740] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.351171  8841 consensus_queue.cc:260] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.351714  8848 consensus_queue.cc:1048] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.354144  8841 ts_tablet_manager.cc:1434] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:15.356719  8892 raft_consensus.cc:493] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.357223  8892 raft_consensus.cc:515] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.359477  8892 leader_election.cc:290] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.360720  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.361377  8797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.362279  8797 raft_consensus.cc:2393] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d0078f73ea741d0b2170b0b88ded4b8 in current term 1: Already voted for candidate eb5c1701174444b2ba83545192501b9c in this term.
I20260709 13:16:15.363369  8614 leader_election.cc:304] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8; no voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c
I20260709 13:16:15.364185  8892 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.376232  8892 raft_consensus.cc:2749] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.376737  8889 raft_consensus.cc:515] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.379685  8889 leader_election.cc:290] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.380649  8649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.381445  8649 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.388026  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.388763  8798 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.402884  8863 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.403417  8649 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.403527  8863 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.406344  8863 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.409281  8689 leader_election.cc:304] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.410089  8798 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:15.410416  8889 raft_consensus.cc:2804] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.411162  8889 raft_consensus.cc:697] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:15.412346  8889 consensus_queue.cc:237] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.416692  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.417467  8798 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 0.
I20260709 13:16:15.418794  8614 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:15.420367  8847 raft_consensus.cc:2804] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.420751  8847 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.421139  8847 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.421341  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.422266  8724 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 0.
I20260709 13:16:15.428122  8847 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.429967  8857 raft_consensus.cc:493] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.430506  8857 raft_consensus.cc:515] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.431654  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:15.431906  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:15.432351  8724 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.432572  8798 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.432811  8857 leader_election.cc:290] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:15.431685  8847 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Requested vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.435367  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" is_pre_election: true
I20260709 13:16:15.437077  8650 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 0.
I20260709 13:16:15.436492  8545 catalog_manager.cc:5697] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 1 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.438372  8763 leader_election.cc:304] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:15.439517  8857 raft_consensus.cc:2804] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.439919  8857 raft_consensus.cc:493] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.440255  8857 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.441679  8724 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 1.
I20260709 13:16:15.442907  8613 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:15.443740  8863 raft_consensus.cc:2804] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.444247  8863 raft_consensus.cc:697] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Running, Role: LEADER
I20260709 13:16:15.443414  8798 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0078f73ea741d0b2170b0b88ded4b8 in term 1.
I20260709 13:16:15.445052  8863 consensus_queue.cc:237] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.436285  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.447865  8857 raft_consensus.cc:515] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.451040  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:15.451579  8650 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.452553  8857 leader_election.cc:290] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:15.447948  8725 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 0.
I20260709 13:16:15.454107  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:15.454785  8725 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.454972  8547 catalog_manager.cc:5697] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.458472  8650 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:15.459586  8763 leader_election.cc:304] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:15.460356  8857 raft_consensus.cc:2804] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.460886  8857 raft_consensus.cc:697] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:15.461826  8857 consensus_queue.cc:237] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.462333  8725 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:15.470628  8547 catalog_manager.cc:5697] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.516865  8863 raft_consensus.cc:493] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.517433  8863 raft_consensus.cc:515] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:15.519236  8863 leader_election.cc:290] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:15.520252  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:15.520265  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a9971ce80048c182443a31df2193c2" candidate_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:15.521243  8798 raft_consensus.cc:2393] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d0078f73ea741d0b2170b0b88ded4b8 in current term 1: Already voted for candidate eb5c1701174444b2ba83545192501b9c in this term.
I20260709 13:16:15.522253  8614 leader_election.cc:304] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8; no voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c
I20260709 13:16:15.523061  8863 raft_consensus.cc:3060] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.528153  8863 raft_consensus.cc:2749] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.608788  8889 consensus_queue.cc:1048] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.621552  8889 consensus_queue.cc:1048] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.635376 32224 ts_manager.cc:295] Set tserver state for 3d0078f73ea741d0b2170b0b88ded4b8 to MAINTENANCE_MODE
I20260709 13:16:15.636539 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:16:15.668370 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:15.670814 32224 tablet_replica.cc:333] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.671556 32224 raft_consensus.cc:2243] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:15.672451 32224 pending_rounds.cc:70] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Trying to abort 1 pending ops.
I20260709 13:16:15.672660 32224 pending_rounds.cc:77] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637787432648704 op_type: NO_OP noop_request { }
I20260709 13:16:15.673031 32224 raft_consensus.cc:2889] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:15.673313 32224 raft_consensus.cc:2272] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.675904 32224 tablet_replica.cc:333] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.676477 32224 raft_consensus.cc:2243] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.676874 32224 raft_consensus.cc:2272] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.678898 32224 tablet_replica.cc:333] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.679481 32224 raft_consensus.cc:2243] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.679925 32224 raft_consensus.cc:2272] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.681559 32224 tablet_replica.cc:333] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.682106 32224 raft_consensus.cc:2243] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.682515 32224 raft_consensus.cc:2272] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.684556 32224 tablet_replica.cc:333] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.685110 32224 raft_consensus.cc:2243] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.685523 32224 raft_consensus.cc:2272] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.687752 32224 tablet_replica.cc:333] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.688310 32224 raft_consensus.cc:2243] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.688714 32224 raft_consensus.cc:2272] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.690893 32224 tablet_replica.cc:333] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.691484 32224 raft_consensus.cc:2243] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:15.692170 32224 pending_rounds.cc:70] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Trying to abort 1 pending ops.
I20260709 13:16:15.692395 32224 pending_rounds.cc:77] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637786683383808 op_type: NO_OP noop_request { }
I20260709 13:16:15.692724 32224 raft_consensus.cc:2889] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:15.693017 32224 raft_consensus.cc:2272] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.695365 32224 tablet_replica.cc:333] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.695950 32224 raft_consensus.cc:2243] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:15.696830 32224 raft_consensus.cc:2272] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.699119 32224 tablet_replica.cc:333] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.699718 32224 raft_consensus.cc:2243] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.700105 32224 raft_consensus.cc:2272] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:15.702317 32224 tablet_replica.cc:333] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: stopping tablet replica
I20260709 13:16:15.702834 32224 raft_consensus.cc:2243] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:15.703195 32224 raft_consensus.cc:2272] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:15.703564  8689 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 203 similar messages]
W20260709 13:16:15.706622  8689 consensus_peers.cc:597] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:15.711345  8889 consensus_queue.cc:1048] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.730556 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
W20260709 13:16:15.736980  8763 consensus_peers.cc:597] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:15.758627  8857 consensus_queue.cc:1048] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:15.791301  8763 consensus_peers.cc:597] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:15.792572  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:16:15.820639  8689 consensus_peers.cc:597] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:15.825551  8846 consensus_queue.cc:1048] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.841717  8889 consensus_queue.cc:1048] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:15.855284  8689 consensus_peers.cc:597] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:15.937776  8763 consensus_peers.cc:597] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:16.039422  8857 consensus_queue.cc:1048] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:16.092278  8689 consensus_peers.cc:597] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:16.738564  8846 raft_consensus.cc:493] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.739077  8846 raft_consensus.cc:515] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.740775  8846 leader_election.cc:290] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:16.742508  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
I20260709 13:16:16.743260  8798 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 1.
I20260709 13:16:16.744364  8689 leader_election.cc:304] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 
W20260709 13:16:16.745078  8689 leader_election.cc:336] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:16.745110  8846 raft_consensus.cc:2804] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:16.745625  8846 raft_consensus.cc:493] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.745908  8846 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.751466  8846 raft_consensus.cc:515] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.753443  8846 leader_election.cc:290] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:16.755057  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:16.755820  8798 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:16.757339  8689 leader_election.cc:336] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:16.762394  8798 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 2.
I20260709 13:16:16.764027  8689 leader_election.cc:304] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:16.765122  8846 raft_consensus.cc:2804] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.765544  8846 raft_consensus.cc:697] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:16.766149  8846 consensus_queue.cc:237] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.773720  8545 catalog_manager.cc:5697] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 1 to 2, leader changed from 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 2 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:16.793660  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:16.883710  8846 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d0078f73ea741d0b2170b0b88ded4b8)
I20260709 13:16:16.884393  8846 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.886325  8846 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:16.887945  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3" is_pre_election: true
W20260709 13:16:16.890008  8689 leader_election.cc:336] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:16.890326  8689 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:16.891186  8846 raft_consensus.cc:2749] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:16.902375  8857 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d0078f73ea741d0b2170b0b88ded4b8)
I20260709 13:16:16.903035  8857 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.905350  8857 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:16.906401  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:16.907137  8725 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:16.908357  8762 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 
W20260709 13:16:16.908978  8763 leader_election.cc:336] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:16.909042  8857 raft_consensus.cc:2804] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:16.909514  8857 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d0078f73ea741d0b2170b0b88ded4b8)
I20260709 13:16:16.909811  8857 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:16.915838  8857 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.917968  8857 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:16.919221  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:16.919870  8725 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:16.921609  8763 leader_election.cc:336] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:16.924944  8725 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:16.926123  8762 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:16.926887  8857 raft_consensus.cc:2804] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:16.927878  8857 raft_consensus.cc:697] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:16.928810  8857 consensus_queue.cc:237] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:16.936136  8545 catalog_manager.cc:5697] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 1 to 2, leader changed from 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.084264  8857 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.084750  8857 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:17.086597  8857 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:17.087734  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c" is_pre_election: true
I20260709 13:16:17.088424  8725 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 1.
I20260709 13:16:17.089429  8762 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:17.090201  8857 raft_consensus.cc:2804] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:17.090461  8763 leader_election.cc:336] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:17.090603  8857 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.091063  8857 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:17.096415  8857 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:17.098181  8857 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:17.099294  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:17.100073  8725 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:17.102367  8763 leader_election.cc:336] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111)
I20260709 13:16:17.105934  8725 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:17.106985  8762 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c5e4b341ca14a52badd1585192b0cb3, eb5c1701174444b2ba83545192501b9c; no voters: 3d0078f73ea741d0b2170b0b88ded4b8
I20260709 13:16:17.107790  8857 raft_consensus.cc:2804] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:17.108217  8857 raft_consensus.cc:697] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:17.108933  8857 consensus_queue.cc:237] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:17.116811  8545 catalog_manager.cc:5697] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 1 to 2, leader changed from 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.149400  8798 raft_consensus.cc:1275] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:17.150750  8846 consensus_queue.cc:1048] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:17.161085  8689 consensus_peers.cc:597] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:17.438458  8763 consensus_peers.cc:597] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:17.524123  8725 raft_consensus.cc:1275] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:17.525529  8857 consensus_queue.cc:1048] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:17.538712  8725 raft_consensus.cc:1275] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:17.540724  8857 consensus_queue.cc:1048] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:17.546535  8763 consensus_peers.cc:597] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:17.794703  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:16:18.129598  8689 consensus_peers.cc:597] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.142416  8763 consensus_peers.cc:597] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.193619  8763 consensus_peers.cc:597] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.280966  8689 consensus_peers.cc:597] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.510793  8689 consensus_peers.cc:597] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.560070  8763 consensus_peers.cc:597] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.567574  8689 consensus_peers.cc:597] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:18.795754  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:16:19.711097  8689 consensus_peers.cc:597] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:19.797053  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:16:19.922322  8763 consensus_peers.cc:597] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:19.942682  8763 consensus_peers.cc:597] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:20.549486  8763 consensus_peers.cc:597] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:20.706487  8689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111) [suppressed 98 similar messages]
W20260709 13:16:20.708855  8689 consensus_peers.cc:597] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:20.754554 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:20.761557  8940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:20.761849  8941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:20.765228  8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:20.765372 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:20.767315 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:20.767517 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:20.767769 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602980767748 us; error 0 us; skew 500 ppm
I20260709 13:16:20.768360 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:20.771023 32224 webserver.cc:533] Webserver started at http://127.31.120.1:40043/ using document root <none> and password file <none>
I20260709 13:16:20.771548 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:20.771863 32224 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:16:20.777511  8689 consensus_peers.cc:597] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:20.779577 32224 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:20.783830  8948 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:20.784647 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:16:20.784945 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
format_stamp: "Formatted at 2026-07-09 13:16:13 on dist-test-slave-pm98"
I20260709 13:16:20.785261 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:16:20.798368  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:20.804636 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
W20260709 13:16:20.806087  8689 consensus_peers.cc:597] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:20.806591 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:20.808228 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:20.811955  8955 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:16:20.845809  8763 consensus_peers.cc:597] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:20.860466 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:16:20.860893 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s	user 0.002s	sys 0.000s
I20260709 13:16:20.861217 32224 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:16:20.865789  8955 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.878290  8955 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:20.879102  8955 tablet_bootstrap.cc:492] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.879755  8955 ts_tablet_manager.cc:1403] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 13:16:20.881690  8955 raft_consensus.cc:359] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.882216  8955 raft_consensus.cc:740] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.882972  8955 consensus_queue.cc:260] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.885121  8955 ts_tablet_manager.cc:1434] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:20.885826  8955 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.898114  8955 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:20.898677 32224 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:16:20.898890  8955 tablet_bootstrap.cc:492] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.899039 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.038s	user 0.032s	sys 0.003s
I20260709 13:16:20.899784  8955 ts_tablet_manager.cc:1403] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.014s	user 0.014s	sys 0.000s
I20260709 13:16:20.901804  8955 raft_consensus.cc:359] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.902331  8955 raft_consensus.cc:740] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.903007  8955 consensus_queue.cc:260] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.904523  8955 ts_tablet_manager.cc:1434] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:16:20.905194  8955 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.918011  8955 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:16:20.918851  8955 tablet_bootstrap.cc:492] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.919453  8955 ts_tablet_manager.cc:1403] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:16:20.922214  8955 raft_consensus.cc:359] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.923166  8955 raft_consensus.cc:740] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.923969  8955 consensus_queue.cc:260] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.925815  8955 ts_tablet_manager.cc:1434] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:20.926630  8955 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.937994  8955 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:16:20.938771  8955 tablet_bootstrap.cc:492] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.939309  8955 ts_tablet_manager.cc:1403] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:16:20.941413  8955 raft_consensus.cc:359] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.942082  8955 raft_consensus.cc:740] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.942823  8955 consensus_queue.cc:260] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.944306  8955 ts_tablet_manager.cc:1434] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:16:20.945053  8955 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.959143 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:37209
I20260709 13:16:20.959270  9017 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:37209 every 8 connection(s)
I20260709 13:16:20.959982  8955 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:20.961018  8955 tablet_bootstrap.cc:492] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.961678  8955 ts_tablet_manager.cc:1403] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:16:20.964076  8955 raft_consensus.cc:359] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:20.964944  8955 raft_consensus.cc:740] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.965466 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:20.965726  8955 consensus_queue.cc:260] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:20.967813  8955 ts_tablet_manager.cc:1434] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:20.968678  8955 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:20.989344  8955 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:20.990283  8955 tablet_bootstrap.cc:492] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:20.992096  8955 ts_tablet_manager.cc:1403] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.000s
I20260709 13:16:20.995561  8955 raft_consensus.cc:359] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.996212  8955 raft_consensus.cc:740] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.996959  8955 consensus_queue.cc:260] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:20.999090  8955 ts_tablet_manager.cc:1434] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:21.000017  8955 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:21.004113  9018 heartbeater.cc:344] Connected to a master server at 127.31.120.62:41135
I20260709 13:16:21.004616  9018 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:21.005144  8798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
mode: GRACEFUL
new_leader_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:49282
I20260709 13:16:21.005611  9018 heartbeater.cc:507] Master 127.31.120.62:41135 requested a full tablet report, sending...
I20260709 13:16:21.005930  8798 raft_consensus.cc:606] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Received request to transfer leadership to TS eb5c1701174444b2ba83545192501b9c
I20260709 13:16:21.016737  8547 ts_manager.cc:194] Re-registered known tserver with Master: 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:21.018113  8955 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:21.019078  8955 tablet_bootstrap.cc:492] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:21.019851  8955 ts_tablet_manager.cc:1403] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:16:21.022064  8725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
mode: GRACEFUL
new_leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:57410
I20260709 13:16:21.022279  8955 raft_consensus.cc:359] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.022969  8955 raft_consensus.cc:740] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.022861  8725 raft_consensus.cc:606] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Received request to transfer leadership to TS 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:21.023852  8955 consensus_queue.cc:260] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.024783 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:21.025550 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:21.025978  8955 ts_tablet_manager.cc:1434] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:16:21.026310 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:21.026834  8955 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:21.029347  8547 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54016
I20260709 13:16:21.032888  9018 heartbeater.cc:499] Master 127.31.120.62:41135 was elected leader, sending a full tablet report...
I20260709 13:16:21.040114  8955 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:21.040830  8955 tablet_bootstrap.cc:492] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:21.041417  8955 ts_tablet_manager.cc:1403] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:16:21.043506  8955 raft_consensus.cc:359] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.044099  8955 raft_consensus.cc:740] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.044871  8991 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.044855  8955 consensus_queue.cc:260] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.047096  8955 ts_tablet_manager.cc:1434] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:21.048045  8955 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:21.053766  8991 pending_rounds.cc:85] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting all ops after (but not including) 0
I20260709 13:16:21.054214  8991 pending_rounds.cc:107] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:16:21.054574  8991 raft_consensus.cc:2889] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.073599  8955 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:21.074318  8955 tablet_bootstrap.cc:492] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:21.074888  8955 ts_tablet_manager.cc:1403] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.012s
I20260709 13:16:21.076985  8955 raft_consensus.cc:359] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.077642  8955 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.078316  8955 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.080427  8955 ts_tablet_manager.cc:1434] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:16:21.081079  8955 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap starting.
I20260709 13:16:21.093647  8955 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:21.094328  8955 tablet_bootstrap.cc:492] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Bootstrap complete.
I20260709 13:16:21.094900  8955 ts_tablet_manager.cc:1403] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:16:21.096750  8955 raft_consensus.cc:359] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.097251  8955 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0078f73ea741d0b2170b0b88ded4b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.097952  8955 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.099799  8955 ts_tablet_manager.cc:1434] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:16:21.172857  8846 raft_consensus.cc:993] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: : Instructing follower 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:21.173277  9035 raft_consensus.cc:1081] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Signalling peer 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:21.174786  8798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:54824
I20260709 13:16:21.175287  8798 raft_consensus.cc:493] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:21.175583  8798 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.182442  8798 raft_consensus.cc:515] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.185582  8798 leader_election.cc:290] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:21.185417  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:21.185763  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59fe8b5187f4b1e90cd8eabf6f74bec" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:21.186642  8990 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.186874  8725 raft_consensus.cc:3055] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:21.187248  8725 raft_consensus.cc:740] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:21.188143  8725 consensus_queue.cc:260] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.189513  8725 raft_consensus.cc:3060] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.194633  8990 raft_consensus.cc:2468] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:21.195932  8763 leader_election.cc:304] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:21.196674  8937 raft_consensus.cc:2804] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:21.196681  8725 raft_consensus.cc:2468] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:21.198026  8937 raft_consensus.cc:697] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:21.198923  8937 consensus_queue.cc:237] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.205832  8547 catalog_manager.cc:5697] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 1 to 2, leader changed from eb5c1701174444b2ba83545192501b9c (127.31.120.2) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:21.302013  8990 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.307104  8990 pending_rounds.cc:85] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting all ops after (but not including) 0
I20260709 13:16:21.307323  8990 pending_rounds.cc:107] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:16:21.307498  8990 raft_consensus.cc:2889] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.435850  8990 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.457547  9037 raft_consensus.cc:993] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: : Instructing follower eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:21.458019  8937 raft_consensus.cc:1081] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Signalling peer eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:21.459529  8725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:59672
I20260709 13:16:21.460093  8725 raft_consensus.cc:493] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:21.460387  8725 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.465366  8725 raft_consensus.cc:515] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.466964  8725 leader_election.cc:290] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:21.468030  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:21.467940  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:21.468575  8990 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.468619  8798 raft_consensus.cc:3055] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:21.469058  8798 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:21.469866  8798 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.470979  8798 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:21.474241  8990 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 2.
I20260709 13:16:21.475528  8689 leader_election.cc:304] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:21.475811  8798 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 2.
I20260709 13:16:21.476380  9035 raft_consensus.cc:2804] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:21.476933  9035 raft_consensus.cc:697] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:21.477753  9035 consensus_queue.cc:237] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:21.485961  8547 catalog_manager.cc:5697] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 1 to 2, leader changed from 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 2 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.748786  8725 raft_consensus.cc:1275] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.750250  9049 consensus_queue.cc:1048] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:21.761127  8990 raft_consensus.cc:1275] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.762696  9037 consensus_queue.cc:1048] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:21.799454  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:21.952574  8798 raft_consensus.cc:1275] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.953964  9048 consensus_queue.cc:1048] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:21.963668  8990 raft_consensus.cc:1275] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.965894  9048 consensus_queue.cc:1048] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.037638 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:22.045774  8798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
mode: GRACEFUL
new_leader_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:49282
I20260709 13:16:22.046336  8798 raft_consensus.cc:606] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Received request to transfer leadership to TS eb5c1701174444b2ba83545192501b9c
I20260709 13:16:22.049048  8725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
mode: GRACEFUL
new_leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:57410
I20260709 13:16:22.049682  8725 raft_consensus.cc:606] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Received request to transfer leadership to TS 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:22.050866 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:22.051512 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:22.052157 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:22.161486  9049 raft_consensus.cc:993] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: : Instructing follower eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:22.162089  9037 raft_consensus.cc:1081] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Signalling peer eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:22.163802  8725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:59672
I20260709 13:16:22.164276  8725 raft_consensus.cc:493] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:22.164535  8725 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.169059  8725 raft_consensus.cc:515] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.170562  8725 leader_election.cc:290] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:22.171361  8991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:22.171905  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aeaf1602891492a80c8c997936bc0ec" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:22.172012  8991 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.172465  8798 raft_consensus.cc:3055] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:22.172744  8798 raft_consensus.cc:740] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:22.173468  8798 consensus_queue.cc:260] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.174556  8798 raft_consensus.cc:3060] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:22.178049  8991 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:22.179105  8689 leader_election.cc:304] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:22.179370  8798 raft_consensus.cc:2468] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:22.179790  9035 raft_consensus.cc:2804] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:22.180377  9035 raft_consensus.cc:697] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:22.181195  9035 consensus_queue.cc:237] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.188349  8547 catalog_manager.cc:5697] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 2 to 3, leader changed from 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 3 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.280468  9048 raft_consensus.cc:993] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: : Instructing follower 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:22.280929  9035 raft_consensus.cc:1081] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Signalling peer 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:22.282439  8798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:54824
I20260709 13:16:22.282912  8798 raft_consensus.cc:493] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:22.283260  8798 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.287777  8798 raft_consensus.cc:515] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.289219  8798 leader_election.cc:290] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:22.290102  8991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:22.290292  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:22.290815  8725 raft_consensus.cc:3055] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:22.290787  8991 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.291509  8725 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:22.292305  8725 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.293458  8725 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.298431  8991 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:22.299736  8763 leader_election.cc:304] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:22.300014  8725 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:22.300470  9037 raft_consensus.cc:2804] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:22.301087  9037 raft_consensus.cc:697] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:22.301944  9037 consensus_queue.cc:237] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:22.308873  8547 catalog_manager.cc:5697] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 1 to 2, leader changed from eb5c1701174444b2ba83545192501b9c (127.31.120.2) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.703709  8798 raft_consensus.cc:1275] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:22.705001  9035 consensus_queue.cc:1048] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.714890  8991 raft_consensus.cc:1275] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:22.717396  9046 consensus_queue.cc:1048] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.774883  8991 raft_consensus.cc:1275] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:22.776306  9037 consensus_queue.cc:1048] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.788288  8725 raft_consensus.cc:1275] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:22.790177  8937 consensus_queue.cc:1048] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:22.800534  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:23.059752 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:23.068321  8798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
mode: GRACEFUL
new_leader_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:49282
I20260709 13:16:23.068953  8798 raft_consensus.cc:606] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Received request to transfer leadership to TS eb5c1701174444b2ba83545192501b9c
I20260709 13:16:23.071872  8724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
mode: GRACEFUL
new_leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:57410
I20260709 13:16:23.072510  8724 raft_consensus.cc:606] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Received request to transfer leadership to TS 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:23.073865 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:23.074580 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:23.075356 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:23.301302  9069 raft_consensus.cc:993] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: : Instructing follower eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:23.301741  9037 raft_consensus.cc:1081] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Signalling peer eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:23.303376  8724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:59672
I20260709 13:16:23.303951  8724 raft_consensus.cc:493] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:23.304270  8724 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.309365  8724 raft_consensus.cc:515] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:23.310990  8724 leader_election.cc:290] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Requested vote from peers 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309), 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209)
I20260709 13:16:23.311913  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:23.312096  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78ecc05ba86a459c855f3ccd2e8f9d71" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:23.312624  8798 raft_consensus.cc:3055] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.312741  8990 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.312911  8798 raft_consensus.cc:740] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:23.313763  8798 consensus_queue.cc:260] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:23.314620  8798 raft_consensus.cc:3060] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.317946  8990 raft_consensus.cc:2468] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 2.
I20260709 13:16:23.319201  8689 leader_election.cc:304] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:23.319486  8798 raft_consensus.cc:2468] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 2.
I20260709 13:16:23.320042  9046 raft_consensus.cc:2804] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.320549  9046 raft_consensus.cc:697] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:23.321374  9046 consensus_queue.cc:237] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } }
I20260709 13:16:23.328011  8545 catalog_manager.cc:5697] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 1 to 2, leader changed from 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 2 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.580490  9035 raft_consensus.cc:993] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: : Instructing follower 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:23.580957  9046 raft_consensus.cc:1081] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Signalling peer 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:23.582686  8798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:54824
I20260709 13:16:23.583155  8798 raft_consensus.cc:493] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:23.583457  8798 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.588164  8798 raft_consensus.cc:515] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:23.589613  8798 leader_election.cc:290] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:23.590682  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:23.590750  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7e64c63b5d4981ab11de3b7b3b3a87" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:23.591650  8990 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.591784  8724 raft_consensus.cc:3055] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:23.592252  8724 raft_consensus.cc:740] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:23.592964  8724 consensus_queue.cc:260] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:23.593931  8724 raft_consensus.cc:3060] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.597950  8990 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:23.599215  8724 raft_consensus.cc:2468] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 2.
I20260709 13:16:23.599264  8763 leader_election.cc:304] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:23.600594  9073 raft_consensus.cc:2804] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.601109  9073 raft_consensus.cc:697] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:23.601852  9073 consensus_queue.cc:237] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:23.609611  8545 catalog_manager.cc:5697] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 1 to 2, leader changed from eb5c1701174444b2ba83545192501b9c (127.31.120.2) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.743211  8798 raft_consensus.cc:1275] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:23.744381  9046 consensus_queue.cc:1048] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:23.755118  8990 raft_consensus.cc:1275] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:23.757898  9046 consensus_queue.cc:1048] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:23.801750  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:24.012630  8724 raft_consensus.cc:1275] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.014185  9073 consensus_queue.cc:1048] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:24.024128  8991 raft_consensus.cc:1275] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:24.025866  9073 consensus_queue.cc:1048] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:24.081941 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:24.090097  8724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10393414464648d3840bc817aa4a70dd"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
mode: GRACEFUL
new_leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:57410
I20260709 13:16:24.090701  8724 raft_consensus.cc:606] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Received request to transfer leadership to TS 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:24.093266  8798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
mode: GRACEFUL
new_leader_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:49282
I20260709 13:16:24.093823  8798 raft_consensus.cc:606] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Received request to transfer leadership to TS eb5c1701174444b2ba83545192501b9c
I20260709 13:16:24.095052 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:24.095644 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:24.096221 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:24.187726  9085 raft_consensus.cc:993] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: : Instructing follower eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:24.188366  9073 raft_consensus.cc:1081] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Signalling peer eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:24.189992  8724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:59672
I20260709 13:16:24.190604  8724 raft_consensus.cc:493] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:24.190936  8724 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.196275  8724 raft_consensus.cc:515] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.197901  8724 leader_election.cc:290] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:24.198848  8991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:24.198907  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74246a9ea1cb44188bf02173c34162fc" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:24.199594  8991 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.199712  8798 raft_consensus.cc:3055] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:24.200135  8798 raft_consensus.cc:740] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:24.200839  8798 consensus_queue.cc:260] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.201707  8798 raft_consensus.cc:3060] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.204818  8991 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:24.206504  8689 leader_election.cc:304] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:24.206506  8798 raft_consensus.cc:2468] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:24.207515  9079 raft_consensus.cc:2804] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:24.208755  9079 raft_consensus.cc:697] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:24.209688  9079 consensus_queue.cc:237] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.219264  8547 catalog_manager.cc:5697] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 2 to 3, leader changed from 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 3 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:24.513238  9046 raft_consensus.cc:993] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: : Instructing follower 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:24.513793  9091 raft_consensus.cc:1081] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Signalling peer 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:24.515275  8798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10393414464648d3840bc817aa4a70dd"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:54824
I20260709 13:16:24.515743  8798 raft_consensus.cc:493] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:24.516052  8798 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.520340  8798 raft_consensus.cc:515] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.521709  8798 leader_election.cc:290] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 3 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:24.522815  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:24.523105  8991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10393414464648d3840bc817aa4a70dd" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:24.523545  8724 raft_consensus.cc:3055] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:24.523691  8991 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.523963  8724 raft_consensus.cc:740] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:24.524811  8724 consensus_queue.cc:260] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.526062  8724 raft_consensus.cc:3060] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:24.531001  8991 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 3.
I20260709 13:16:24.532588  8763 leader_election.cc:304] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:24.533253  9073 raft_consensus.cc:2804] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:24.533205  8724 raft_consensus.cc:2468] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 3.
I20260709 13:16:24.534010  9073 raft_consensus.cc:697] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:24.534878  9073 consensus_queue.cc:237] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:24.541989  8547 catalog_manager.cc:5697] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 2 to 3, leader changed from eb5c1701174444b2ba83545192501b9c (127.31.120.2) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.699501  8798 raft_consensus.cc:1275] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:24.699718  8991 raft_consensus.cc:1275] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:24.702250  9046 consensus_queue.cc:1048] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:24.703449  9091 consensus_queue.cc:1048] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:24.802948  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:25.011689  8990 raft_consensus.cc:1275] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:25.013043  9085 consensus_queue.cc:1048] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.025187  8724 raft_consensus.cc:1275] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:25.027035  9085 consensus_queue.cc:1048] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.102679 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:25.111920  8725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
mode: GRACEFUL
new_leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:57410
I20260709 13:16:25.112524  8725 raft_consensus.cc:606] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Received request to transfer leadership to TS 4c5e4b341ca14a52badd1585192b0cb3
I20260709 13:16:25.115625  8798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
mode: GRACEFUL
new_leader_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:49282
I20260709 13:16:25.116182  8798 raft_consensus.cc:606] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Received request to transfer leadership to TS eb5c1701174444b2ba83545192501b9c
I20260709 13:16:25.117700 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:25.118229 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:25.119022 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:25.439894  9102 raft_consensus.cc:993] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: : Instructing follower 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:25.440514  9102 raft_consensus.cc:1081] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Signalling peer 4c5e4b341ca14a52badd1585192b0cb3 to start an election
I20260709 13:16:25.441943  8798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef"
dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
 from {username='slave'} at 127.0.0.1:54824
I20260709 13:16:25.442538  8798 raft_consensus.cc:493] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:25.442898  8798 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.450794  8798 raft_consensus.cc:515] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.452585  8798 leader_election.cc:290] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 3 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227)
I20260709 13:16:25.453879  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:25.454169  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c0657dddef485fb8df5e83964123ef" candidate_uuid: "4c5e4b341ca14a52badd1585192b0cb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb5c1701174444b2ba83545192501b9c"
I20260709 13:16:25.454753  8990 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.454804  8725 raft_consensus.cc:3055] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:25.455258  8725 raft_consensus.cc:740] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:25.456163  8725 consensus_queue.cc:260] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.457360  8725 raft_consensus.cc:3060] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.462836  8990 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 3.
I20260709 13:16:25.464262  8725 raft_consensus.cc:2468] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c5e4b341ca14a52badd1585192b0cb3 in term 3.
I20260709 13:16:25.464413  8763 leader_election.cc:304] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, 4c5e4b341ca14a52badd1585192b0cb3; no voters: 
I20260709 13:16:25.465693  9073 raft_consensus.cc:2804] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:25.466213  9073 raft_consensus.cc:697] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 LEADER]: Becoming Leader. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:25.467164  9073 consensus_queue.cc:237] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.475776  8545 catalog_manager.cc:5697] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 reported cstate change: term changed from 2 to 3, leader changed from eb5c1701174444b2ba83545192501b9c (127.31.120.2) to 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "4c5e4b341ca14a52badd1585192b0cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.491044  9085 raft_consensus.cc:993] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: : Instructing follower eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:25.491540  9073 raft_consensus.cc:1081] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Signalling peer eb5c1701174444b2ba83545192501b9c to start an election
I20260709 13:16:25.493091  8725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f"
dest_uuid: "eb5c1701174444b2ba83545192501b9c"
 from {username='slave'} at 127.0.0.1:59672
I20260709 13:16:25.493523  8725 raft_consensus.cc:493] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:25.493811  8725 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.498652  8725 raft_consensus.cc:515] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.500226  8725 leader_election.cc:290] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Requested vote from peers 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209), 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3:33309)
I20260709 13:16:25.501008  8990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3d0078f73ea741d0b2170b0b88ded4b8"
I20260709 13:16:25.501148  8798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea636fd0e5648a883d808804e8ffc0f" candidate_uuid: "eb5c1701174444b2ba83545192501b9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c5e4b341ca14a52badd1585192b0cb3"
I20260709 13:16:25.501716  8990 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.501761  8798 raft_consensus.cc:3055] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:25.502156  8798 raft_consensus.cc:740] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c5e4b341ca14a52badd1585192b0cb3, State: Running, Role: LEADER
I20260709 13:16:25.502978  8798 consensus_queue.cc:260] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.503938  8798 raft_consensus.cc:3060] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:25.507452  8990 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:25.508709  8689 leader_election.cc:304] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0078f73ea741d0b2170b0b88ded4b8, eb5c1701174444b2ba83545192501b9c; no voters: 
I20260709 13:16:25.509402  9102 raft_consensus.cc:2804] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:25.509389  8798 raft_consensus.cc:2468] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c1701174444b2ba83545192501b9c in term 3.
I20260709 13:16:25.510219  9102 raft_consensus.cc:697] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Becoming Leader. State: Replica: eb5c1701174444b2ba83545192501b9c, State: Running, Role: LEADER
I20260709 13:16:25.511253  9102 consensus_queue.cc:237] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } }
I20260709 13:16:25.518577  8545 catalog_manager.cc:5697] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c reported cstate change: term changed from 2 to 3, leader changed from 4c5e4b341ca14a52badd1585192b0cb3 (127.31.120.3) to eb5c1701174444b2ba83545192501b9c (127.31.120.2). New cstate: current_term: 3 leader_uuid: "eb5c1701174444b2ba83545192501b9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:25.804014  8596 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:25.919706  8725 raft_consensus.cc:1275] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:25.921190  9108 consensus_queue.cc:1048] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5c1701174444b2ba83545192501b9c" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:25.934244  8990 raft_consensus.cc:1275] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Refusing update from remote peer 4c5e4b341ca14a52badd1585192b0cb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:25.935560  9108 consensus_queue.cc:1048] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:25.962438  8990 raft_consensus.cc:1275] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:25.963667  9102 consensus_queue.cc:1048] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0078f73ea741d0b2170b0b88ded4b8" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:25.973482  8798 raft_consensus.cc:1275] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Refusing update from remote peer eb5c1701174444b2ba83545192501b9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:25.975311  9079 consensus_queue.cc:1048] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c5e4b341ca14a52badd1585192b0cb3" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:26.125237 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:26.156594 32224 tablet_server.cc:179] TabletServer@127.31.120.1:37209 shutting down...
I20260709 13:16:26.190204 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:26.191176 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.192191 32224 raft_consensus.cc:2243] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.192759 32224 raft_consensus.cc:2272] T 04c0657dddef485fb8df5e83964123ef P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.196626 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.197602 32224 raft_consensus.cc:2243] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.198268 32224 raft_consensus.cc:2272] T 879a556aa631478fa439a825c5529391 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:26.199724  8689 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
I20260709 13:16:26.201709 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.202615 32224 raft_consensus.cc:2243] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:26.202890  8689 consensus_peers.cc:597] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:26.203191 32224 raft_consensus.cc:2272] T 74246a9ea1cb44188bf02173c34162fc P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.206207 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.206911 32224 raft_consensus.cc:2243] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.207464 32224 raft_consensus.cc:2272] T b59fe8b5187f4b1e90cd8eabf6f74bec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.210204 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.210834 32224 raft_consensus.cc:2243] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.211345 32224 raft_consensus.cc:2272] T b7a9971ce80048c182443a31df2193c2 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.213891 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.214486 32224 raft_consensus.cc:2243] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.214970 32224 raft_consensus.cc:2272] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.217458 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.218016 32224 raft_consensus.cc:2243] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.218458 32224 raft_consensus.cc:2272] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 3d0078f73ea741d0b2170b0b88ded4b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.220799 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.221362 32224 raft_consensus.cc:2243] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.221801 32224 raft_consensus.cc:2272] T aea636fd0e5648a883d808804e8ffc0f P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.224706 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.225260 32224 raft_consensus.cc:2243] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.225711 32224 raft_consensus.cc:2272] T 10393414464648d3840bc817aa4a70dd P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.227917 32224 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:26.228428 32224 raft_consensus.cc:2243] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.228909 32224 raft_consensus.cc:2272] T 2aeaf1602891492a80c8c997936bc0ec P 3d0078f73ea741d0b2170b0b88ded4b8 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:26.247822  8952 connection.cc:466] server connection from 127.0.0.1:60354 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60354 (request call id 61) could send its response
W20260709 13:16:26.248672  8763 consensus_peers.cc:597] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: recv got EOF from 127.31.120.1:37209 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:26.263175 32224 tablet_server.cc:196] TabletServer@127.31.120.1:37209 shutdown complete.
I20260709 13:16:26.292896 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:16:26.318521  8689 consensus_peers.cc:597] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c -> Peer 3d0078f73ea741d0b2170b0b88ded4b8 (127.31.120.1:37209): Couldn't send request to peer 3d0078f73ea741d0b2170b0b88ded4b8. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:26.321631 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:26.322444 32224 tablet_replica.cc:333] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.323163 32224 raft_consensus.cc:2243] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.323659 32224 raft_consensus.cc:2272] T 04c0657dddef485fb8df5e83964123ef P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.327071 32224 tablet_replica.cc:333] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.327864 32224 raft_consensus.cc:2243] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.328612 32224 raft_consensus.cc:2272] T b59fe8b5187f4b1e90cd8eabf6f74bec P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.330838 32224 tablet_replica.cc:333] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.331465 32224 raft_consensus.cc:2243] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.332340 32224 raft_consensus.cc:2272] T 74246a9ea1cb44188bf02173c34162fc P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.334599 32224 tablet_replica.cc:333] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.335405 32224 raft_consensus.cc:2243] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.336470 32224 raft_consensus.cc:2272] T aea636fd0e5648a883d808804e8ffc0f P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:26.339223  8762 consensus_peers.cc:597] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 -> Peer eb5c1701174444b2ba83545192501b9c (127.31.120.2:36227): Couldn't send request to peer eb5c1701174444b2ba83545192501b9c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:26.340461 32224 tablet_replica.cc:333] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.341070 32224 raft_consensus.cc:2243] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.341496 32224 raft_consensus.cc:2272] T 10393414464648d3840bc817aa4a70dd P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.343824 32224 tablet_replica.cc:333] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.344336 32224 raft_consensus.cc:2243] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.345144 32224 raft_consensus.cc:2272] T b7a9971ce80048c182443a31df2193c2 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.347292 32224 tablet_replica.cc:333] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.347820 32224 raft_consensus.cc:2243] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.348557 32224 raft_consensus.cc:2272] T 78ecc05ba86a459c855f3ccd2e8f9d71 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.350525 32224 tablet_replica.cc:333] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.351001 32224 raft_consensus.cc:2243] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.351444 32224 raft_consensus.cc:2272] T bd7e64c63b5d4981ab11de3b7b3b3a87 P eb5c1701174444b2ba83545192501b9c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.353642 32224 tablet_replica.cc:333] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.354132 32224 raft_consensus.cc:2243] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.354967 32224 raft_consensus.cc:2272] T 2aeaf1602891492a80c8c997936bc0ec P eb5c1701174444b2ba83545192501b9c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.357074 32224 tablet_replica.cc:333] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c: stopping tablet replica
I20260709 13:16:26.357559 32224 raft_consensus.cc:2243] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.358012 32224 raft_consensus.cc:2272] T 879a556aa631478fa439a825c5529391 P eb5c1701174444b2ba83545192501b9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.383206 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
I20260709 13:16:26.409538 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:16:26.437405 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:26.438146 32224 tablet_replica.cc:333] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.438918 32224 raft_consensus.cc:2243] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.439831 32224 raft_consensus.cc:2272] T 04c0657dddef485fb8df5e83964123ef P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.442865 32224 tablet_replica.cc:333] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.443435 32224 raft_consensus.cc:2243] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.443962 32224 raft_consensus.cc:2272] T 2aeaf1602891492a80c8c997936bc0ec P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.446112 32224 tablet_replica.cc:333] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.446664 32224 raft_consensus.cc:2243] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.447085 32224 raft_consensus.cc:2272] T aea636fd0e5648a883d808804e8ffc0f P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.449970 32224 tablet_replica.cc:333] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.450644 32224 raft_consensus.cc:2243] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.451686 32224 raft_consensus.cc:2272] T 10393414464648d3840bc817aa4a70dd P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.453835 32224 tablet_replica.cc:333] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.454442 32224 raft_consensus.cc:2243] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.455312 32224 raft_consensus.cc:2272] T 879a556aa631478fa439a825c5529391 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.457674 32224 tablet_replica.cc:333] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.458230 32224 raft_consensus.cc:2243] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.458701 32224 raft_consensus.cc:2272] T b7a9971ce80048c182443a31df2193c2 P 4c5e4b341ca14a52badd1585192b0cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.461455 32224 tablet_replica.cc:333] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.462052 32224 raft_consensus.cc:2243] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.462565 32224 raft_consensus.cc:2272] T 78ecc05ba86a459c855f3ccd2e8f9d71 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.464851 32224 tablet_replica.cc:333] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.465394 32224 raft_consensus.cc:2243] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.466265 32224 raft_consensus.cc:2272] T bd7e64c63b5d4981ab11de3b7b3b3a87 P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.468441 32224 tablet_replica.cc:333] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.468922 32224 raft_consensus.cc:2243] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.469337 32224 raft_consensus.cc:2272] T 74246a9ea1cb44188bf02173c34162fc P 4c5e4b341ca14a52badd1585192b0cb3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.471403 32224 tablet_replica.cc:333] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3: stopping tablet replica
I20260709 13:16:26.471940 32224 raft_consensus.cc:2243] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.472821 32224 raft_consensus.cc:2272] T b59fe8b5187f4b1e90cd8eabf6f74bec P 4c5e4b341ca14a52badd1585192b0cb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.505738 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:16:26.534031 32224 master.cc:562] Master@127.31.120.62:41135 shutting down...
I20260709 13:16:26.558422 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:26.559059 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.559454 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P 732fdcdd3a714d76b8e1268e327c4b6f: stopping tablet replica
I20260709 13:16:26.580583 32224 master.cc:584] Master@127.31.120.62:41135 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12997 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:16:26.630353 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:39083
I20260709 13:16:26.631783 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:26.637988  9123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:26.638588  9124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:26.639439 32224 server_base.cc:1061] running on GCE node
W20260709 13:16:26.640120  9126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:26.641158 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:26.641520 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:26.641709 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602986641672 us; error 0 us; skew 500 ppm
I20260709 13:16:26.642344 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:26.645195 32224 webserver.cc:533] Webserver started at http://127.31.120.62:44221/ using document root <none> and password file <none>
I20260709 13:16:26.645758 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:26.645982 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:26.646296 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:26.647544 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "3186f660356a42788ef82d5e25f39d6a"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.652658 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:16:26.656306  9131 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:26.657255 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:16:26.657632 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "3186f660356a42788ef82d5e25f39d6a"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.657930 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:26.676288 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:26.677529 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:26.716106 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:39083
I20260709 13:16:26.716203  9182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:39083 every 8 connection(s)
I20260709 13:16:26.720216  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:26.731892  9183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a: Bootstrap starting.
I20260709 13:16:26.736711  9183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.741077  9183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a: No bootstrap required, opened a new log
I20260709 13:16:26.743399  9183 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER }
I20260709 13:16:26.743839  9183 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.744028  9183 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3186f660356a42788ef82d5e25f39d6a, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.744650  9183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER }
I20260709 13:16:26.745055  9183 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:26.745244  9183 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:26.745457  9183 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.750226  9183 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER }
I20260709 13:16:26.750690  9183 leader_election.cc:304] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3186f660356a42788ef82d5e25f39d6a; no voters: 
I20260709 13:16:26.751837  9183 leader_election.cc:290] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:26.752243  9186 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.753620  9186 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 1 LEADER]: Becoming Leader. State: Replica: 3186f660356a42788ef82d5e25f39d6a, State: Running, Role: LEADER
I20260709 13:16:26.754307  9186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER }
I20260709 13:16:26.754834  9183 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:26.759193  9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3186f660356a42788ef82d5e25f39d6a. Latest consensus state: current_term: 1 leader_uuid: "3186f660356a42788ef82d5e25f39d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER } }
I20260709 13:16:26.759996  9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:26.761703  9187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3186f660356a42788ef82d5e25f39d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3186f660356a42788ef82d5e25f39d6a" member_type: VOTER } }
I20260709 13:16:26.762097  9193 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:26.762225  9187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:26.771296  9193 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:26.777335 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:26.782295  9193 catalog_manager.cc:1383] Generated new cluster ID: eda0a8f4db664a37889c9416e2ba62ab
I20260709 13:16:26.782595  9193 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:26.802274  9193 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:26.803586  9193 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:26.826457  9193 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a: Generated new TSK 0
I20260709 13:16:26.827148  9193 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:26.843894 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:26.850728  9204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:26.851349  9205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:26.853132  9207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:26.853691 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:26.854660 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:26.854904 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:26.855083 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602986855065 us; error 0 us; skew 500 ppm
I20260709 13:16:26.855744 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:26.858215 32224 webserver.cc:533] Webserver started at http://127.31.120.1:42215/ using document root <none> and password file <none>
I20260709 13:16:26.858705 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:26.858920 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:26.859185 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:26.860462 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "b6f14d31855f484a808aa7cad4e0ba85"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.865203 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:16:26.868944  9212 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:26.869920 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:16:26.870188 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "b6f14d31855f484a808aa7cad4e0ba85"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.870446 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:26.886240 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:26.887408 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:26.888911 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:26.891346 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:26.891525 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:26.891852 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:26.892035 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:26.934178 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:43257
I20260709 13:16:26.934361  9274 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:43257 every 8 connection(s)
I20260709 13:16:26.939776 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:26.947075  9279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:26.947435  9280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:26.951020  9282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:26.952704  9275 heartbeater.cc:344] Connected to a master server at 127.31.120.62:39083
I20260709 13:16:26.952934 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:26.953258  9275 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:26.954264  9275 heartbeater.cc:507] Master 127.31.120.62:39083 requested a full tablet report, sending...
I20260709 13:16:26.954385 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:26.954746 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:26.954959 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602986954943 us; error 0 us; skew 500 ppm
I20260709 13:16:26.955648 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:26.956810  9148 ts_manager.cc:194] Registered new tserver with Master: b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257)
I20260709 13:16:26.958688  9148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46196
I20260709 13:16:26.958761 32224 webserver.cc:533] Webserver started at http://127.31.120.2:45991/ using document root <none> and password file <none>
I20260709 13:16:26.959496 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:26.959751 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:26.960125 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:26.961767 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "1e812e642910454fb0abfe343c4d6055"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.966261 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:26.970037  9287 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:26.970899 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:16:26.971241 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "1e812e642910454fb0abfe343c4d6055"
format_stamp: "Formatted at 2026-07-09 13:16:26 on dist-test-slave-pm98"
I20260709 13:16:26.971532 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:27.026952 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:27.028205 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:27.029637 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:27.031952 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:27.032130 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:27.032485 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:27.032634 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:27.074827 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:35287
I20260709 13:16:27.074981  9349 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:35287 every 8 connection(s)
I20260709 13:16:27.079844 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:27.087178  9353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:27.089370  9354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:27.092475 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:27.093789  9350 heartbeater.cc:344] Connected to a master server at 127.31.120.62:39083
W20260709 13:16:27.093892  9356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:27.094413  9350 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:27.095232 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:27.095626 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:27.095688  9350 heartbeater.cc:507] Master 127.31.120.62:39083 requested a full tablet report, sending...
I20260709 13:16:27.095944 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602987095921 us; error 0 us; skew 500 ppm
I20260709 13:16:27.096909 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:27.098461  9148 ts_manager.cc:194] Registered new tserver with Master: 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.099761 32224 webserver.cc:533] Webserver started at http://127.31.120.3:42191/ using document root <none> and password file <none>
I20260709 13:16:27.100184  9148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46202
I20260709 13:16:27.100310 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:27.100663 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:27.100941 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:27.102432 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "6e55b35d4802416ea199a8c893a0bf27"
format_stamp: "Formatted at 2026-07-09 13:16:27 on dist-test-slave-pm98"
I20260709 13:16:27.107188 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:27.111056  9361 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:27.112154 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:16:27.112542 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "6e55b35d4802416ea199a8c893a0bf27"
format_stamp: "Formatted at 2026-07-09 13:16:27 on dist-test-slave-pm98"
I20260709 13:16:27.112807 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:27.133621 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:27.134794 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:27.136484 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:27.138692 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:27.138918 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:27.139170 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:27.139343 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:27.181272 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:41301
I20260709 13:16:27.181438  9423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:41301 every 8 connection(s)
I20260709 13:16:27.196231  9424 heartbeater.cc:344] Connected to a master server at 127.31.120.62:39083
I20260709 13:16:27.196770  9424 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:27.197548  9424 heartbeater.cc:507] Master 127.31.120.62:39083 requested a full tablet report, sending...
I20260709 13:16:27.199760  9148 ts_manager.cc:194] Registered new tserver with Master: 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301)
I20260709 13:16:27.200022 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015157924s
I20260709 13:16:27.201887  9148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46212
I20260709 13:16:27.228957  9148 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46222:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:27.231066  9148 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:27.285558  9240 tablet_service.cc:1511] Processing CreateTablet for tablet 5a10418eb5794073b86d5f351405b138 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.287119  9240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a10418eb5794073b86d5f351405b138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.290748  9389 tablet_service.cc:1511] Processing CreateTablet for tablet 5a10418eb5794073b86d5f351405b138 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.292712  9389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a10418eb5794073b86d5f351405b138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.292209  9239 tablet_service.cc:1511] Processing CreateTablet for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.294059  9239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.305773  9315 tablet_service.cc:1511] Processing CreateTablet for tablet 5a10418eb5794073b86d5f351405b138 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.307057  9314 tablet_service.cc:1511] Processing CreateTablet for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.305791  9388 tablet_service.cc:1511] Processing CreateTablet for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=c2dc23b0ba3c499e905047466a1f0f05]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.309629  9388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.311903  9315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a10418eb5794073b86d5f351405b138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.312772  9314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a7d6ab3c1da4c369e0607fdb728eb4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.325086  9439 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: Bootstrap starting.
I20260709 13:16:27.337307  9440 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: Bootstrap starting.
I20260709 13:16:27.341696  9439 tablet_bootstrap.cc:654] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.344178  9441 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: Bootstrap starting.
I20260709 13:16:27.345077  9440 tablet_bootstrap.cc:654] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.349777  9439 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: No bootstrap required, opened a new log
I20260709 13:16:27.350306  9439 ts_tablet_manager.cc:1403] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:16:27.350498  9440 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: No bootstrap required, opened a new log
I20260709 13:16:27.351096  9440 ts_tablet_manager.cc:1403] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:16:27.352244  9441 tablet_bootstrap.cc:654] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.353752  9440 raft_consensus.cc:359] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.353555  9439 raft_consensus.cc:359] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.354523  9440 raft_consensus.cc:385] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.354621  9439 raft_consensus.cc:385] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.354984  9440 raft_consensus.cc:740] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.355064  9439 raft_consensus.cc:740] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.356069  9440 consensus_queue.cc:260] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.356112  9439 consensus_queue.cc:260] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.360936  9424 heartbeater.cc:499] Master 127.31.120.62:39083 was elected leader, sending a full tablet report...
I20260709 13:16:27.361375  9440 ts_tablet_manager.cc:1434] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:16:27.362013  9275 heartbeater.cc:499] Master 127.31.120.62:39083 was elected leader, sending a full tablet report...
I20260709 13:16:27.361375  9439 ts_tablet_manager.cc:1434] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.003s
I20260709 13:16:27.362811  9440 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: Bootstrap starting.
I20260709 13:16:27.363792  9439 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: Bootstrap starting.
I20260709 13:16:27.368907  9441 tablet_bootstrap.cc:492] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: No bootstrap required, opened a new log
I20260709 13:16:27.369591  9441 ts_tablet_manager.cc:1403] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.004s
I20260709 13:16:27.370880  9440 tablet_bootstrap.cc:654] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.371799  9439 tablet_bootstrap.cc:654] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.373428  9441 raft_consensus.cc:359] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.374331  9441 raft_consensus.cc:385] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.374686  9441 raft_consensus.cc:740] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e812e642910454fb0abfe343c4d6055, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.375998  9441 consensus_queue.cc:260] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.382695  9440 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: No bootstrap required, opened a new log
I20260709 13:16:27.383320  9440 ts_tablet_manager.cc:1403] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.010s
I20260709 13:16:27.383620  9439 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: No bootstrap required, opened a new log
I20260709 13:16:27.384490  9439 ts_tablet_manager.cc:1403] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:16:27.387964  9350 heartbeater.cc:499] Master 127.31.120.62:39083 was elected leader, sending a full tablet report...
I20260709 13:16:27.387831  9440 raft_consensus.cc:359] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.388072  9439 raft_consensus.cc:359] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.388681  9440 raft_consensus.cc:385] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.388545  9441 ts_tablet_manager.cc:1434] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.005s
I20260709 13:16:27.388967  9439 raft_consensus.cc:385] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.389137  9440 raft_consensus.cc:740] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.389432  9439 raft_consensus.cc:740] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.390089  9441 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Bootstrap starting.
I20260709 13:16:27.390254  9440 consensus_queue.cc:260] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.390612  9439 consensus_queue.cc:260] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.393144  9439 ts_tablet_manager.cc:1434] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:27.396847  9440 ts_tablet_manager.cc:1434] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:16:27.397953  9441 tablet_bootstrap.cc:654] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.403376  9446 raft_consensus.cc:493] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.404023  9446 raft_consensus.cc:515] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.405516  9441 tablet_bootstrap.cc:492] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: No bootstrap required, opened a new log
I20260709 13:16:27.406071  9441 ts_tablet_manager.cc:1403] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:16:27.408224  9446 leader_election.cc:290] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.409301  9441 raft_consensus.cc:359] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.410143  9441 raft_consensus.cc:385] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.410486  9441 raft_consensus.cc:740] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e812e642910454fb0abfe343c4d6055, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.411285  9441 consensus_queue.cc:260] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.413673  9441 ts_tablet_manager.cc:1434] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:27.420408  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85" is_pre_election: true
I20260709 13:16:27.421247  9250 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.422766  9363 leader_election.cc:304] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.423733  9446 raft_consensus.cc:2804] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.423975  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055" is_pre_election: true
I20260709 13:16:27.424196  9446 raft_consensus.cc:493] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.424723  9446 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.424794  9325 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.431015  9446 raft_consensus.cc:515] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.432730  9446 leader_election.cc:290] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Requested vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.433722  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
I20260709 13:16:27.433841  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:27.434280  9250 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.434453  9325 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.440434  9250 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.440508  9325 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.441754  9363 leader_election.cc:304] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.442629  9446 raft_consensus.cc:2804] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.443797  9446 raft_consensus.cc:697] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Leader. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:27.444651  9446 consensus_queue.cc:237] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:27.452068  9146 catalog_manager.cc:5697] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "6e55b35d4802416ea199a8c893a0bf27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.585994  9446 raft_consensus.cc:493] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.586563  9446 raft_consensus.cc:515] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.588549  9446 leader_election.cc:290] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.589418  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85" is_pre_election: true
I20260709 13:16:27.589608  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055" is_pre_election: true
I20260709 13:16:27.590240  9250 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.590255  9325 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.591540  9363 leader_election.cc:304] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.592315  9446 raft_consensus.cc:2804] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.592717  9446 raft_consensus.cc:493] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.593116  9446 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.598284  9446 raft_consensus.cc:515] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.600078  9446 leader_election.cc:290] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Requested vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.600841  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
I20260709 13:16:27.601056  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:27.601441  9250 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.601814  9325 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.606588  9250 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.606788  9325 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.607952  9363 leader_election.cc:304] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.608706  9446 raft_consensus.cc:2804] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.609135  9446 raft_consensus.cc:697] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Leader. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:27.609822  9446 consensus_queue.cc:237] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.616421  9146 catalog_manager.cc:5697] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "6e55b35d4802416ea199a8c893a0bf27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:27.684288  9146 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46232:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:27.686506  9146 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:27.709306  9240 tablet_service.cc:1511] Processing CreateTablet for tablet c46b075bc40c4327a5d651293ebf16e8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.710608  9240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46b075bc40c4327a5d651293ebf16e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.710702  9315 tablet_service.cc:1511] Processing CreateTablet for tablet c46b075bc40c4327a5d651293ebf16e8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.712095  9315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46b075bc40c4327a5d651293ebf16e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.714322  9239 tablet_service.cc:1511] Processing CreateTablet for tablet c8bed896068d419d9b21e4082fec77a4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.714686  9314 tablet_service.cc:1511] Processing CreateTablet for tablet c8bed896068d419d9b21e4082fec77a4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.715941  9314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bed896068d419d9b21e4082fec77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.714406  9388 tablet_service.cc:1511] Processing CreateTablet for tablet c46b075bc40c4327a5d651293ebf16e8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:16:27.716986  9388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46b075bc40c4327a5d651293ebf16e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.717290  9389 tablet_service.cc:1511] Processing CreateTablet for tablet c8bed896068d419d9b21e4082fec77a4 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e44b7fd18a2d44f48c170e6477202f74]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:16:27.718531  9389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bed896068d419d9b21e4082fec77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.720405  9239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8bed896068d419d9b21e4082fec77a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:27.732261  9441 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: Bootstrap starting.
I20260709 13:16:27.734889  9440 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: Bootstrap starting.
I20260709 13:16:27.737484  9439 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: Bootstrap starting.
I20260709 13:16:27.743044  9440 tablet_bootstrap.cc:654] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.746131  9441 tablet_bootstrap.cc:654] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.747004  9439 tablet_bootstrap.cc:654] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.758780  9441 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: No bootstrap required, opened a new log
I20260709 13:16:27.758803  9440 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: No bootstrap required, opened a new log
I20260709 13:16:27.758962  9439 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: No bootstrap required, opened a new log
I20260709 13:16:27.759493  9441 ts_tablet_manager.cc:1403] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.006s
I20260709 13:16:27.759517  9440 ts_tablet_manager.cc:1403] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.000s
I20260709 13:16:27.759759  9439 ts_tablet_manager.cc:1403] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.003s
I20260709 13:16:27.763145  9439 raft_consensus.cc:359] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.763056  9441 raft_consensus.cc:359] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.763566  9440 raft_consensus.cc:359] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.764060  9439 raft_consensus.cc:385] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.764158  9441 raft_consensus.cc:385] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.764304  9440 raft_consensus.cc:385] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.764474  9439 raft_consensus.cc:740] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.764727  9441 raft_consensus.cc:740] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e812e642910454fb0abfe343c4d6055, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.764770  9440 raft_consensus.cc:740] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.765529  9439 consensus_queue.cc:260] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.765803  9441 consensus_queue.cc:260] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.765945  9440 consensus_queue.cc:260] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.767755  9439 ts_tablet_manager.cc:1434] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:27.769321  9439 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Bootstrap starting.
I20260709 13:16:27.770020  9441 ts_tablet_manager.cc:1434] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:16:27.772691  9441 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: Bootstrap starting.
W20260709 13:16:27.773839  9196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:27.775799  9440 ts_tablet_manager.cc:1434] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:16:27.776975  9440 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Bootstrap starting.
I20260709 13:16:27.782684  9441 tablet_bootstrap.cc:654] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.784008  9440 tablet_bootstrap.cc:654] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.791730  9439 tablet_bootstrap.cc:654] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.794481  9441 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: No bootstrap required, opened a new log
I20260709 13:16:27.795082  9441 ts_tablet_manager.cc:1403] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:16:27.795213  9440 tablet_bootstrap.cc:492] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: No bootstrap required, opened a new log
I20260709 13:16:27.795797  9440 ts_tablet_manager.cc:1403] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.003s
I20260709 13:16:27.797070  9439 tablet_bootstrap.cc:492] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: No bootstrap required, opened a new log
I20260709 13:16:27.797616  9439 ts_tablet_manager.cc:1403] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:16:27.798148  9441 raft_consensus.cc:359] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.798947  9441 raft_consensus.cc:385] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.799235  9441 raft_consensus.cc:740] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e812e642910454fb0abfe343c4d6055, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.800072  9441 consensus_queue.cc:260] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.800709  9440 raft_consensus.cc:359] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.801019  9439 raft_consensus.cc:359] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.801610  9440 raft_consensus.cc:385] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.801726  9439 raft_consensus.cc:385] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.802021  9440 raft_consensus.cc:740] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.802157  9441 ts_tablet_manager.cc:1434] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:27.802115  9439 raft_consensus.cc:740] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.802974  9440 consensus_queue.cc:260] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.803395  9439 consensus_queue.cc:260] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.806006  9440 ts_tablet_manager.cc:1434] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:16:27.808745  9439 ts_tablet_manager.cc:1434] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 13:16:27.889074  9445 raft_consensus.cc:493] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.889719  9445 raft_consensus.cc:515] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.892059  9445 leader_election.cc:290] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287), 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301)
I20260709 13:16:27.902664  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055" is_pre_election: true
I20260709 13:16:27.903354  9325 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 0.
I20260709 13:16:27.904433  9215 leader_election.cc:304] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e812e642910454fb0abfe343c4d6055, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.904631  9399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e55b35d4802416ea199a8c893a0bf27" is_pre_election: true
I20260709 13:16:27.905228  9445 raft_consensus.cc:2804] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.905536  9399 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 0.
I20260709 13:16:27.905684  9445 raft_consensus.cc:493] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.906121  9445 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.911988  9445 raft_consensus.cc:515] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.913676  9445 leader_election.cc:290] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 1 election: Requested vote from peers 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287), 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301)
I20260709 13:16:27.914525  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:27.914748  9399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
I20260709 13:16:27.915266  9325 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.915345  9399 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.923544  9325 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 1.
I20260709 13:16:27.923712  9399 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 1.
I20260709 13:16:27.924919  9214 leader_election.cc:304] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.925632  9445 raft_consensus.cc:2804] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.926640  9445 raft_consensus.cc:697] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 LEADER]: Becoming Leader. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Running, Role: LEADER
I20260709 13:16:27.927461  9445 consensus_queue.cc:237] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.934470  9148 catalog_manager.cc:5697] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "b6f14d31855f484a808aa7cad4e0ba85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.957187  9446 raft_consensus.cc:493] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.957710  9446 raft_consensus.cc:515] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.959472  9446 leader_election.cc:290] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.960544  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85" is_pre_election: true
I20260709 13:16:27.960623  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055" is_pre_election: true
I20260709 13:16:27.961282  9250 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.961337  9325 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 0.
I20260709 13:16:27.962483  9363 leader_election.cc:304] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:27.963259  9446 raft_consensus.cc:2804] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.963735  9446 raft_consensus.cc:493] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.964061  9446 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.969357  9446 raft_consensus.cc:515] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.971434  9446 leader_election.cc:290] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Requested vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:27.972404  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
I20260709 13:16:27.972666  9325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:27.973189  9250 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.973272  9325 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.978367  9325 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.978511  9250 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 1.
I20260709 13:16:27.979487  9364 leader_election.cc:304] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e812e642910454fb0abfe343c4d6055, 6e55b35d4802416ea199a8c893a0bf27; no voters: 
I20260709 13:16:27.980293  9446 raft_consensus.cc:2804] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.980845  9446 raft_consensus.cc:697] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Leader. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:27.981580  9446 consensus_queue.cc:237] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:27.990345  9148 catalog_manager.cc:5697] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "6e55b35d4802416ea199a8c893a0bf27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:27.991353  9446 consensus_queue.cc:1048] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.992993  9446 consensus_queue.cc:1048] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.011050  9446 consensus_queue.cc:1048] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.021926  9446 consensus_queue.cc:1048] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.060418  9399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a10418eb5794073b86d5f351405b138"
dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
mode: GRACEFUL
new_leader_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
 from {username='slave'} at 127.0.0.1:36976
I20260709 13:16:28.061081  9399 raft_consensus.cc:606] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Received request to transfer leadership to TS b6f14d31855f484a808aa7cad4e0ba85
I20260709 13:16:28.067070  9399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f"
dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
mode: GRACEFUL
new_leader_uuid: "1e812e642910454fb0abfe343c4d6055"
 from {username='slave'} at 127.0.0.1:36976
I20260709 13:16:28.067677  9399 raft_consensus.cc:606] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Received request to transfer leadership to TS 1e812e642910454fb0abfe343c4d6055
I20260709 13:16:28.075913  9399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4"
dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
mode: GRACEFUL
new_leader_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
 from {username='slave'} at 127.0.0.1:36976
I20260709 13:16:28.076447  9399 raft_consensus.cc:606] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Received request to transfer leadership to TS b6f14d31855f484a808aa7cad4e0ba85
I20260709 13:16:28.362743  9445 consensus_queue.cc:1048] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.376315  9445 consensus_queue.cc:1048] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.419843  9446 raft_consensus.cc:993] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: : Instructing follower b6f14d31855f484a808aa7cad4e0ba85 to start an election
I20260709 13:16:28.420401  9452 raft_consensus.cc:1081] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Signalling peer b6f14d31855f484a808aa7cad4e0ba85 to start an election
I20260709 13:16:28.421980  9250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a10418eb5794073b86d5f351405b138"
dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
 from {username='slave'} at 127.0.0.1:33228
I20260709 13:16:28.422613  9250 raft_consensus.cc:493] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:28.423041  9250 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.428270  9250 raft_consensus.cc:515] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:28.430037  9250 leader_election.cc:290] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 2 election: Requested vote from peers 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:28.430944  9399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
I20260709 13:16:28.431185  9324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a10418eb5794073b86d5f351405b138" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:28.431536  9399 raft_consensus.cc:3055] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:28.431877  9399 raft_consensus.cc:740] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:28.431936  9324 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.432562  9399 consensus_queue.cc:260] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:28.433475  9399 raft_consensus.cc:3060] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.437945  9324 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 2.
I20260709 13:16:28.438596  9399 raft_consensus.cc:2468] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 2.
I20260709 13:16:28.439308  9215 leader_election.cc:304] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e812e642910454fb0abfe343c4d6055, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:28.440300  9445 raft_consensus.cc:2804] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:28.440840  9445 raft_consensus.cc:697] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 LEADER]: Becoming Leader. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Running, Role: LEADER
I20260709 13:16:28.441818  9445 consensus_queue.cc:237] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } }
I20260709 13:16:28.448115  9148 catalog_manager.cc:5697] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 reported cstate change: term changed from 1 to 2, leader changed from 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3) to b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "b6f14d31855f484a808aa7cad4e0ba85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.585654  9452 consensus_queue.cc:1048] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.596684  9475 consensus_queue.cc:1048] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.603905  9446 raft_consensus.cc:993] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: : Instructing follower b6f14d31855f484a808aa7cad4e0ba85 to start an election
I20260709 13:16:28.604521  9475 raft_consensus.cc:1081] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Signalling peer b6f14d31855f484a808aa7cad4e0ba85 to start an election
I20260709 13:16:28.606071  9250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4"
dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
 from {username='slave'} at 127.0.0.1:33228
I20260709 13:16:28.606626  9250 raft_consensus.cc:493] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:28.607002  9250 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.612228  9250 raft_consensus.cc:515] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.613857  9250 leader_election.cc:290] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 2 election: Requested vote from peers 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287), 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301)
I20260709 13:16:28.615034  9324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:28.615036  9399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8bed896068d419d9b21e4082fec77a4" candidate_uuid: "b6f14d31855f484a808aa7cad4e0ba85" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
I20260709 13:16:28.615922  9324 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.616063  9399 raft_consensus.cc:3055] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:28.616503  9399 raft_consensus.cc:740] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:28.617430  9399 consensus_queue.cc:260] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.618523  9399 raft_consensus.cc:3060] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.621119  9324 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 2.
I20260709 13:16:28.622100  9215 leader_election.cc:304] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e812e642910454fb0abfe343c4d6055, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:28.622792  9445 raft_consensus.cc:2804] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:28.623277  9445 raft_consensus.cc:697] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 LEADER]: Becoming Leader. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Running, Role: LEADER
I20260709 13:16:28.623585  9399 raft_consensus.cc:2468] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f14d31855f484a808aa7cad4e0ba85 in term 2.
I20260709 13:16:28.624173  9445 consensus_queue.cc:237] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.630817  9446 raft_consensus.cc:993] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: : Instructing follower 1e812e642910454fb0abfe343c4d6055 to start an election
I20260709 13:16:28.631276  9475 raft_consensus.cc:1081] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Signalling peer 1e812e642910454fb0abfe343c4d6055 to start an election
I20260709 13:16:28.631207  9146 catalog_manager.cc:5697] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 reported cstate change: term changed from 1 to 2, leader changed from 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3) to b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "b6f14d31855f484a808aa7cad4e0ba85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.632985  9324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f"
dest_uuid: "1e812e642910454fb0abfe343c4d6055"
 from {username='slave'} at 127.0.0.1:37968
I20260709 13:16:28.633461  9324 raft_consensus.cc:493] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:28.633836  9324 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.640179  9324 raft_consensus.cc:515] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.642052  9324 leader_election.cc:290] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [CANDIDATE]: Term 2 election: Requested vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3:41301)
I20260709 13:16:28.652979  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "1e812e642910454fb0abfe343c4d6055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
I20260709 13:16:28.653558  9250 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.655022  9399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a7d6ab3c1da4c369e0607fdb728eb4f" candidate_uuid: "1e812e642910454fb0abfe343c4d6055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
I20260709 13:16:28.655575  9399 raft_consensus.cc:3055] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:28.655912  9399 raft_consensus.cc:740] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:28.656607  9399 consensus_queue.cc:260] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.657552  9399 raft_consensus.cc:3060] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.659335  9250 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e812e642910454fb0abfe343c4d6055 in term 2.
I20260709 13:16:28.660781  9289 leader_election.cc:304] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e812e642910454fb0abfe343c4d6055, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:28.661721  9449 raft_consensus.cc:2804] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:28.662988  9449 raft_consensus.cc:697] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 LEADER]: Becoming Leader. State: Replica: 1e812e642910454fb0abfe343c4d6055, State: Running, Role: LEADER
I20260709 13:16:28.663158  9399 raft_consensus.cc:2468] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e812e642910454fb0abfe343c4d6055 in term 2.
I20260709 13:16:28.663882  9449 consensus_queue.cc:237] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.669735  9146 catalog_manager.cc:5697] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 reported cstate change: term changed from 1 to 2, leader changed from 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3) to 1e812e642910454fb0abfe343c4d6055 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "1e812e642910454fb0abfe343c4d6055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.792734 32224 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:16:28.794165 32224 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:16:28.794495 32224 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:16:28.805992  9250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8"
dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
mode: GRACEFUL
new_leader_uuid: "6e55b35d4802416ea199a8c893a0bf27"
 from {username='slave'} at 127.0.0.1:33236
I20260709 13:16:28.806713  9250 raft_consensus.cc:606] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 LEADER]: Received request to transfer leadership to TS 6e55b35d4802416ea199a8c893a0bf27
I20260709 13:16:28.808383 32224 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:16:28.851138  9324 raft_consensus.cc:1275] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Refusing update from remote peer b6f14d31855f484a808aa7cad4e0ba85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:28.852397  9445 consensus_queue.cc:1048] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:28.860424  9399 raft_consensus.cc:1275] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Refusing update from remote peer b6f14d31855f484a808aa7cad4e0ba85: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:28.862350  9486 consensus_queue.cc:1048] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:28.927066  9469 raft_consensus.cc:993] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: : Instructing follower 6e55b35d4802416ea199a8c893a0bf27 to start an election
I20260709 13:16:28.927567  9486 raft_consensus.cc:1081] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 LEADER]: Signalling peer 6e55b35d4802416ea199a8c893a0bf27 to start an election
I20260709 13:16:28.929194  9399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8"
dest_uuid: "6e55b35d4802416ea199a8c893a0bf27"
 from {username='slave'} at 127.0.0.1:36972
I20260709 13:16:28.929729  9399 raft_consensus.cc:493] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:28.930043  9399 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.935065  9399 raft_consensus.cc:515] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.936712  9399 leader_election.cc:290] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 2 election: Requested vote from peers b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1:43257), 1e812e642910454fb0abfe343c4d6055 (127.31.120.2:35287)
I20260709 13:16:28.937902  9250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6f14d31855f484a808aa7cad4e0ba85"
I20260709 13:16:28.938483  9250 raft_consensus.cc:3055] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:28.938769  9250 raft_consensus.cc:740] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6f14d31855f484a808aa7cad4e0ba85, State: Running, Role: LEADER
I20260709 13:16:28.939455  9250 consensus_queue.cc:260] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.940430  9250 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.942085  9324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46b075bc40c4327a5d651293ebf16e8" candidate_uuid: "6e55b35d4802416ea199a8c893a0bf27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e812e642910454fb0abfe343c4d6055"
I20260709 13:16:28.942808  9324 raft_consensus.cc:3060] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.946570  9250 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 2.
I20260709 13:16:28.947746  9363 leader_election.cc:304] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e55b35d4802416ea199a8c893a0bf27, b6f14d31855f484a808aa7cad4e0ba85; no voters: 
I20260709 13:16:28.948469  9475 raft_consensus.cc:2804] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:28.948975  9475 raft_consensus.cc:697] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 LEADER]: Becoming Leader. State: Replica: 6e55b35d4802416ea199a8c893a0bf27, State: Running, Role: LEADER
I20260709 13:16:28.949790  9475 consensus_queue.cc:237] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } }
I20260709 13:16:28.950973  9324 raft_consensus.cc:2468] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e55b35d4802416ea199a8c893a0bf27 in term 2.
I20260709 13:16:28.956703  9146 catalog_manager.cc:5697] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 reported cstate change: term changed from 1 to 2, leader changed from b6f14d31855f484a808aa7cad4e0ba85 (127.31.120.1) to 6e55b35d4802416ea199a8c893a0bf27 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "6e55b35d4802416ea199a8c893a0bf27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6f14d31855f484a808aa7cad4e0ba85" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e812e642910454fb0abfe343c4d6055" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e55b35d4802416ea199a8c893a0bf27" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 41301 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.024995 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:16:29.045753 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:29.046417 32224 tablet_replica.cc:333] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: stopping tablet replica
I20260709 13:16:29.047000 32224 raft_consensus.cc:2243] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:29.047676 32224 pending_rounds.cc:70] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Trying to abort 1 pending ops.
I20260709 13:16:29.047866 32224 pending_rounds.cc:77] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305637841413382144 op_type: NO_OP noop_request { }
I20260709 13:16:29.048128 32224 raft_consensus.cc:2889] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:29.048429 32224 raft_consensus.cc:2272] T c8bed896068d419d9b21e4082fec77a4 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.050649 32224 tablet_replica.cc:333] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85: stopping tablet replica
I20260709 13:16:29.051247 32224 raft_consensus.cc:2243] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.051667 32224 raft_consensus.cc:2272] T 2a7d6ab3c1da4c369e0607fdb728eb4f P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.054307 32224 tablet_replica.cc:333] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85: stopping tablet replica
I20260709 13:16:29.054751 32224 raft_consensus.cc:2243] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.055073 32224 raft_consensus.cc:2272] T c46b075bc40c4327a5d651293ebf16e8 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.057325 32224 tablet_replica.cc:333] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85: stopping tablet replica
I20260709 13:16:29.057803 32224 raft_consensus.cc:2243] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:29.058518 32224 raft_consensus.cc:2272] T 5a10418eb5794073b86d5f351405b138 P b6f14d31855f484a808aa7cad4e0ba85 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.071816 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
I20260709 13:16:29.084661 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
I20260709 13:16:29.102239 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:29.102895 32224 tablet_replica.cc:333] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055: stopping tablet replica
I20260709 13:16:29.103463 32224 raft_consensus.cc:2243] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.103987 32224 raft_consensus.cc:2272] T c8bed896068d419d9b21e4082fec77a4 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.106535 32224 tablet_replica.cc:333] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: stopping tablet replica
I20260709 13:16:29.107057 32224 raft_consensus.cc:2243] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:29.107739 32224 pending_rounds.cc:70] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Trying to abort 1 pending ops.
I20260709 13:16:29.107954 32224 pending_rounds.cc:77] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305637841575211008 op_type: NO_OP noop_request { }
I20260709 13:16:29.108227 32224 raft_consensus.cc:2889] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:29.108541 32224 raft_consensus.cc:2272] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.110662 32224 tablet_replica.cc:333] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055: stopping tablet replica
I20260709 13:16:29.111164 32224 raft_consensus.cc:2243] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.111644 32224 raft_consensus.cc:2272] T c46b075bc40c4327a5d651293ebf16e8 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.114084 32224 tablet_replica.cc:333] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055: stopping tablet replica
I20260709 13:16:29.114619 32224 raft_consensus.cc:2243] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.115048 32224 raft_consensus.cc:2272] T 5a10418eb5794073b86d5f351405b138 P 1e812e642910454fb0abfe343c4d6055 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.140558 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
I20260709 13:16:29.152393 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:16:29.169867 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:29.170578 32224 tablet_replica.cc:333] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27: stopping tablet replica
I20260709 13:16:29.171252 32224 raft_consensus.cc:2243] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.171857 32224 raft_consensus.cc:2272] T c8bed896068d419d9b21e4082fec77a4 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.174152 32224 tablet_replica.cc:333] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27: stopping tablet replica
I20260709 13:16:29.174726 32224 raft_consensus.cc:2243] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.175144 32224 raft_consensus.cc:2272] T 2a7d6ab3c1da4c369e0607fdb728eb4f P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.177001 32224 tablet_replica.cc:333] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: stopping tablet replica
I20260709 13:16:29.177500 32224 raft_consensus.cc:2243] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:29.178093 32224 pending_rounds.cc:70] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Trying to abort 1 pending ops.
I20260709 13:16:29.178285 32224 pending_rounds.cc:77] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305637842748915712 op_type: NO_OP noop_request { }
I20260709 13:16:29.178537 32224 raft_consensus.cc:2889] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:29.178838 32224 raft_consensus.cc:2272] T c46b075bc40c4327a5d651293ebf16e8 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.181526 32224 tablet_replica.cc:333] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27: stopping tablet replica
I20260709 13:16:29.182029 32224 raft_consensus.cc:2243] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:29.182454 32224 raft_consensus.cc:2272] T 5a10418eb5794073b86d5f351405b138 P 6e55b35d4802416ea199a8c893a0bf27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.205353 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:16:29.217629 32224 master.cc:562] Master@127.31.120.62:39083 shutting down...
I20260709 13:16:29.235267 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:29.235878 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:29.236208 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3186f660356a42788ef82d5e25f39d6a: stopping tablet replica
I20260709 13:16:29.255304 32224 master.cc:584] Master@127.31.120.62:39083 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2654 ms)
[----------] 6 tests from LeaderRebalancerTest (274058 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:16:29.285044 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:32987
I20260709 13:16:29.286136 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:29.291344  9496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.291507  9497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.293176  9499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:29.294307 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:29.295186 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:29.295485 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:29.295742 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602989295723 us; error 0 us; skew 500 ppm
I20260709 13:16:29.296327 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:29.298933 32224 webserver.cc:533] Webserver started at http://127.31.120.62:43957/ using document root <none> and password file <none>
I20260709 13:16:29.299871 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:29.300082 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:29.300386 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:29.301506 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "3eb040a608864513ac4117b392d009d6"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.306174 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:29.309769  9504 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.310683 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:29.310966 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "3eb040a608864513ac4117b392d009d6"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.311198 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:29.332403 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:29.333454 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:29.368042 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:32987
I20260709 13:16:29.368140  9555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:32987 every 8 connection(s)
I20260709 13:16:29.371778  9556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.382606  9556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: Bootstrap starting.
I20260709 13:16:29.387061  9556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.391124  9556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: No bootstrap required, opened a new log
I20260709 13:16:29.393163  9556 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER }
I20260709 13:16:29.393533  9556 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.393705  9556 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb040a608864513ac4117b392d009d6, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.394232  9556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER }
I20260709 13:16:29.394619  9556 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:29.394806  9556 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:29.395017  9556 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.399535  9556 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER }
I20260709 13:16:29.400020  9556 leader_election.cc:304] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eb040a608864513ac4117b392d009d6; no voters: 
I20260709 13:16:29.401265  9556 leader_election.cc:290] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:29.401662  9559 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.403203  9559 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3eb040a608864513ac4117b392d009d6, State: Running, Role: LEADER
I20260709 13:16:29.403872  9559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER }
I20260709 13:16:29.404438  9556 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:29.406782  9560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eb040a608864513ac4117b392d009d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER } }
I20260709 13:16:29.406868  9561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eb040a608864513ac4117b392d009d6. Latest consensus state: current_term: 1 leader_uuid: "3eb040a608864513ac4117b392d009d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eb040a608864513ac4117b392d009d6" member_type: VOTER } }
I20260709 13:16:29.407647  9560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:29.407716  9561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:29.410386  9565 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:29.415668  9565 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:29.420958 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:29.424540  9565 catalog_manager.cc:1383] Generated new cluster ID: a716056a79054da28d8fc46e1f3068ac
I20260709 13:16:29.424813  9565 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:29.444344  9565 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:29.445641  9565 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:29.458031  9565 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: Generated new TSK 0
I20260709 13:16:29.458655  9565 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:29.488214 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:29.494673  9577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.496380  9578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.497263  9580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:29.497642 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:29.498565 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:29.498742 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:29.498965 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602989498929 us; error 0 us; skew 500 ppm
I20260709 13:16:29.499518 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:29.501957 32224 webserver.cc:533] Webserver started at http://127.31.120.1:34137/ using document root <none> and password file <none>
I20260709 13:16:29.502462 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:29.502648 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:29.502954 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:29.504108 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "bdf041618a254a1a9d5a2f014162cf90"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.508812 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:16:29.512293  9585 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.513126 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:16:29.513432 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "bdf041618a254a1a9d5a2f014162cf90"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.513731 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:29.530296 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:29.531448 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:29.532895 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:29.535126 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:29.535342 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.535583 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:29.535843 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:16:29.574183 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:40853
I20260709 13:16:29.574303  9647 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:40853 every 8 connection(s)
I20260709 13:16:29.579316 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:29.586134  9652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.587387  9653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.589951  9655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:29.591714  9648 heartbeater.cc:344] Connected to a master server at 127.31.120.62:32987
I20260709 13:16:29.591872 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:29.592255  9648 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:29.592995  9648 heartbeater.cc:507] Master 127.31.120.62:32987 requested a full tablet report, sending...
I20260709 13:16:29.593485 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:29.593716 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:29.593956 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602989593936 us; error 0 us; skew 500 ppm
I20260709 13:16:29.594694 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:29.595124  9521 ts_manager.cc:194] Registered new tserver with Master: bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853)
I20260709 13:16:29.597254 32224 webserver.cc:533] Webserver started at http://127.31.120.2:37625/ using document root <none> and password file <none>
I20260709 13:16:29.597379  9521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58648
I20260709 13:16:29.597872 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:29.598119 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:29.598410 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:29.599946 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "c1e5cfbaa43444d1b773156f512e2d11"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.604240 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:16:29.607427  9660 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.608240 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:16:29.608548 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "c1e5cfbaa43444d1b773156f512e2d11"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.608806 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:29.624769 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:29.625824 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:29.627279 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:29.629451 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:29.629647 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.629912 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:29.630072 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.668656 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:35081
I20260709 13:16:29.668785  9722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:35081 every 8 connection(s)
I20260709 13:16:29.673698 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:29.680853  9726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:29.681916  9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:29.684981 32224 server_base.cc:1061] running on GCE node
W20260709 13:16:29.685580  9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:29.686278  9723 heartbeater.cc:344] Connected to a master server at 127.31.120.62:32987
I20260709 13:16:29.686574 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:16:29.686689  9723 heartbeater.cc:461] Registering TS with master...
W20260709 13:16:29.686897 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:29.687188 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783602989687170 us; error 0 us; skew 500 ppm
I20260709 13:16:29.687692  9723 heartbeater.cc:507] Master 127.31.120.62:32987 requested a full tablet report, sending...
I20260709 13:16:29.687865 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:29.690173  9521 ts_manager.cc:194] Registered new tserver with Master: c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:29.690984 32224 webserver.cc:533] Webserver started at http://127.31.120.3:43283/ using document root <none> and password file <none>
I20260709 13:16:29.691751 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:29.691996 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:29.692176  9521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58652
I20260709 13:16:29.692322 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:29.693862 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "ac386aefbbe344588dea3a68164db507"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.698177 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:16:29.701665  9734 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.702484 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:29.702780 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "ac386aefbbe344588dea3a68164db507"
format_stamp: "Formatted at 2026-07-09 13:16:29 on dist-test-slave-pm98"
I20260709 13:16:29.703105 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:29.716718 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:29.717787 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:29.719187 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:29.721540 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:29.721778 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.722031 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:29.722205 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:29.761890 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:42851
I20260709 13:16:29.762009  9796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:42851 every 8 connection(s)
I20260709 13:16:29.775851  9797 heartbeater.cc:344] Connected to a master server at 127.31.120.62:32987
I20260709 13:16:29.776479  9797 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:29.777212  9797 heartbeater.cc:507] Master 127.31.120.62:32987 requested a full tablet report, sending...
I20260709 13:16:29.779171  9521 ts_manager.cc:194] Registered new tserver with Master: ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:29.779697 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014673124s
I20260709 13:16:29.780634  9521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58656
I20260709 13:16:29.805256  9521 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58664:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:29.807904  9521 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:29.907294  9609 tablet_service.cc:1511] Processing CreateTablet for tablet 43319ce311c54054990dde238848975d (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:29.908857  9609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43319ce311c54054990dde238848975d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.908876  9610 tablet_service.cc:1511] Processing CreateTablet for tablet 96316755af8c4c9dac15c7556b76e4cc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:29.907280  9613 tablet_service.cc:1511] Processing CreateTablet for tablet cc40916a38ed4de5adbc19eec6314c4e (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:29.908902  9611 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc8fd062d764891b4df49bafb99de16 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:29.910568  9610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96316755af8c4c9dac15c7556b76e4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.908902  9612 tablet_service.cc:1511] Processing CreateTablet for tablet e40a69de06904e52a6642505b5a42db1 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:29.911495  9613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc40916a38ed4de5adbc19eec6314c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.911624  9608 tablet_service.cc:1511] Processing CreateTablet for tablet 5a4299f51a58415a93d06cf2685ea63a (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:29.912349  9612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40a69de06904e52a6642505b5a42db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.913350  9608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a4299f51a58415a93d06cf2685ea63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.914131  9611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc8fd062d764891b4df49bafb99de16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.915664  9607 tablet_service.cc:1511] Processing CreateTablet for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:29.917325  9607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.919392  9606 tablet_service.cc:1511] Processing CreateTablet for tablet 5455f6674b914b5f87bdd3652e6a65bf (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:29.921028  9606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5455f6674b914b5f87bdd3652e6a65bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.931684  9605 tablet_service.cc:1511] Processing CreateTablet for tablet d1787d4051c14afcac84c33f9d4ff691 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:29.933286  9605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1787d4051c14afcac84c33f9d4ff691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.939443  9688 tablet_service.cc:1511] Processing CreateTablet for tablet cc40916a38ed4de5adbc19eec6314c4e (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:29.941326  9688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc40916a38ed4de5adbc19eec6314c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.943949  9687 tablet_service.cc:1511] Processing CreateTablet for tablet e40a69de06904e52a6642505b5a42db1 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:29.945920  9687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40a69de06904e52a6642505b5a42db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.949616  9686 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc8fd062d764891b4df49bafb99de16 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:29.951074  9686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc8fd062d764891b4df49bafb99de16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.965193  9685 tablet_service.cc:1511] Processing CreateTablet for tablet 96316755af8c4c9dac15c7556b76e4cc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:29.966589  9685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96316755af8c4c9dac15c7556b76e4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.967763  9762 tablet_service.cc:1511] Processing CreateTablet for tablet cc40916a38ed4de5adbc19eec6314c4e (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:29.969358  9762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc40916a38ed4de5adbc19eec6314c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.970156  9761 tablet_service.cc:1511] Processing CreateTablet for tablet e40a69de06904e52a6642505b5a42db1 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:29.971786  9761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40a69de06904e52a6642505b5a42db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.972833  9757 tablet_service.cc:1511] Processing CreateTablet for tablet 5a4299f51a58415a93d06cf2685ea63a (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:29.974342  9757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a4299f51a58415a93d06cf2685ea63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.976907  9758 tablet_service.cc:1511] Processing CreateTablet for tablet 43319ce311c54054990dde238848975d (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:29.977047  9684 tablet_service.cc:1511] Processing CreateTablet for tablet 43319ce311c54054990dde238848975d (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:29.978601  9758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43319ce311c54054990dde238848975d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.979205  9684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43319ce311c54054990dde238848975d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.980808  9759 tablet_service.cc:1511] Processing CreateTablet for tablet 96316755af8c4c9dac15c7556b76e4cc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:29.982404  9759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96316755af8c4c9dac15c7556b76e4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.984769  9760 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc8fd062d764891b4df49bafb99de16 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:29.989183  9683 tablet_service.cc:1511] Processing CreateTablet for tablet 5a4299f51a58415a93d06cf2685ea63a (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:29.990788  9683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a4299f51a58415a93d06cf2685ea63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.996212  9760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc8fd062d764891b4df49bafb99de16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.001005  9756 tablet_service.cc:1511] Processing CreateTablet for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:30.001767  9754 tablet_service.cc:1511] Processing CreateTablet for tablet d1787d4051c14afcac84c33f9d4ff691 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:30.002712  9756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.003418  9754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1787d4051c14afcac84c33f9d4ff691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.004761  9682 tablet_service.cc:1511] Processing CreateTablet for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:30.006121  9682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3baf9fcdf364ef0b6d9fbe7e4b22dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.006317  9755 tablet_service.cc:1511] Processing CreateTablet for tablet 5455f6674b914b5f87bdd3652e6a65bf (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:30.007869  9755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5455f6674b914b5f87bdd3652e6a65bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.021772  9681 tablet_service.cc:1511] Processing CreateTablet for tablet 5455f6674b914b5f87bdd3652e6a65bf (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:30.023307  9681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5455f6674b914b5f87bdd3652e6a65bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.024837  9680 tablet_service.cc:1511] Processing CreateTablet for tablet d1787d4051c14afcac84c33f9d4ff691 (DEFAULT_TABLE table=test-workload [id=03b7b613ff1440bf94947dcb75a22e75]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:30.039966  9680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1787d4051c14afcac84c33f9d4ff691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:30.101845  9813 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.106281  9812 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.106683  9814 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.113940  9812 tablet_bootstrap.cc:654] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.114562  9814 tablet_bootstrap.cc:654] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.127097  9813 tablet_bootstrap.cc:654] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.129241  9812 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.129792  9812 ts_tablet_manager.cc:1403] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:16:30.133783  9812 raft_consensus.cc:359] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.134547  9812 raft_consensus.cc:385] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.134856  9812 raft_consensus.cc:740] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.135651  9812 consensus_queue.cc:260] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.137250  9814 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.137982  9814 ts_tablet_manager.cc:1403] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.004s
I20260709 13:16:30.141904  9814 raft_consensus.cc:359] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.142961  9814 raft_consensus.cc:385] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.143419  9814 raft_consensus.cc:740] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.144613  9814 consensus_queue.cc:260] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.148259  9797 heartbeater.cc:499] Master 127.31.120.62:32987 was elected leader, sending a full tablet report...
I20260709 13:16:30.148542  9812 ts_tablet_manager.cc:1434] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.009s
I20260709 13:16:30.149580  9812 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.149744  9813 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.150321  9813 ts_tablet_manager.cc:1403] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.012s
I20260709 13:16:30.154975  9648 heartbeater.cc:499] Master 127.31.120.62:32987 was elected leader, sending a full tablet report...
I20260709 13:16:30.155211  9812 tablet_bootstrap.cc:654] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.155524  9814 ts_tablet_manager.cc:1434] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:16:30.156598  9814 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.162686  9814 tablet_bootstrap.cc:654] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.163805  9813 raft_consensus.cc:359] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.164743  9813 raft_consensus.cc:385] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.165057  9813 raft_consensus.cc:740] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.166057  9813 consensus_queue.cc:260] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.167152  9812 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.167778  9812 ts_tablet_manager.cc:1403] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.003s
I20260709 13:16:30.168644  9723 heartbeater.cc:499] Master 127.31.120.62:32987 was elected leader, sending a full tablet report...
I20260709 13:16:30.170727  9813 ts_tablet_manager.cc:1434] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:16:30.170706  9812 raft_consensus.cc:359] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.171695  9812 raft_consensus.cc:385] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.172021  9813 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.172058  9812 raft_consensus.cc:740] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.173851  9812 consensus_queue.cc:260] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.176375  9812 ts_tablet_manager.cc:1434] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:30.177474  9812 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.182700  9814 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.183404  9814 ts_tablet_manager.cc:1403] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:16:30.184005  9812 tablet_bootstrap.cc:654] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.185894  9814 raft_consensus.cc:359] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.186542  9814 raft_consensus.cc:385] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.186839  9814 raft_consensus.cc:740] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.187587  9814 consensus_queue.cc:260] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.192853  9813 tablet_bootstrap.cc:654] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.189791  9814 ts_tablet_manager.cc:1434] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:16:30.195487  9814 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.205955  9814 tablet_bootstrap.cc:654] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.212284  9813 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.212993  9813 ts_tablet_manager.cc:1403] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.011s
I20260709 13:16:30.216223  9812 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.216112  9813 raft_consensus.cc:359] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.216825  9812 ts_tablet_manager.cc:1403] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.025s
I20260709 13:16:30.216922  9813 raft_consensus.cc:385] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.217368  9813 raft_consensus.cc:740] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.217454  9814 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.218066  9814 ts_tablet_manager.cc:1403] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 13:16:30.218195  9813 consensus_queue.cc:260] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.219550  9812 raft_consensus.cc:359] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.220345  9812 raft_consensus.cc:385] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.220490  9813 ts_tablet_manager.cc:1434] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:16:30.220713  9812 raft_consensus.cc:740] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.221052  9814 raft_consensus.cc:359] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.221598  9813 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.221858  9814 raft_consensus.cc:385] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.221671  9812 consensus_queue.cc:260] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.222419  9814 raft_consensus.cc:740] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.223448  9814 consensus_queue.cc:260] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.224665  9819 raft_consensus.cc:493] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.225237  9819 raft_consensus.cc:515] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.227221  9812 ts_tablet_manager.cc:1434] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:16:30.227960  9819 leader_election.cc:290] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.228375  9812 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.228797  9813 tablet_bootstrap.cc:654] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.229866  9818 raft_consensus.cc:493] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.230371  9818 raft_consensus.cc:515] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.233259  9818 leader_election.cc:290] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.238512  9814 ts_tablet_manager.cc:1434] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:30.239436  9812 tablet_bootstrap.cc:654] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.243768  9818 raft_consensus.cc:493] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.244320  9818 raft_consensus.cc:515] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.246542  9818 leader_election.cc:290] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.248304  9814 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.253899  9812 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.254617  9812 ts_tablet_manager.cc:1403] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.000s
I20260709 13:16:30.254474  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.256229  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.257176  9622 raft_consensus.cc:2468] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:30.258570  9812 raft_consensus.cc:359] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.259374  9812 raft_consensus.cc:385] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.259774  9812 raft_consensus.cc:740] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.260392  9814 tablet_bootstrap.cc:654] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.260735  9812 consensus_queue.cc:260] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.262009  9738 leader_election.cc:304] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
W20260709 13:16:30.263144  9738 leader_election.cc:343] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.263792  9818 raft_consensus.cc:2804] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.264184  9818 raft_consensus.cc:493] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.264528  9818 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.273856  9814 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.274737  9814 ts_tablet_manager.cc:1403] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.013s
I20260709 13:16:30.276746  9818 raft_consensus.cc:515] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.280536  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:30.281416  9623 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.289988  9818 leader_election.cc:290] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.279193  9814 raft_consensus.cc:359] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.290809  9814 raft_consensus.cc:385] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.290694  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.293183  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.296128  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
W20260709 13:16:30.298393  9736 leader_election.cc:343] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.299219  9736 leader_election.cc:304] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac386aefbbe344588dea3a68164db507; no voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:30.297937  9814 raft_consensus.cc:740] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.302872  9818 raft_consensus.cc:2749] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.302970  9814 consensus_queue.cc:260] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.306962  9812 ts_tablet_manager.cc:1434] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.052s	user 0.016s	sys 0.003s
W20260709 13:16:30.309165  9736 leader_election.cc:343] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.311327  9623 raft_consensus.cc:2468] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
W20260709 13:16:30.312062  9736 leader_election.cc:343] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.312814  9738 leader_election.cc:304] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:30.313591  9812 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.317526  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.321725  9812 tablet_bootstrap.cc:654] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.311309  9814 ts_tablet_manager.cc:1434] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:16:30.321625  9832 raft_consensus.cc:2804] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.323776  9832 raft_consensus.cc:697] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:30.324770  9832 consensus_queue.cc:237] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
W20260709 13:16:30.321683  9587 leader_election.cc:343] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.331637  9814 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.330662  9813 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.335276  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.336261  9813 ts_tablet_manager.cc:1403] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.115s	user 0.047s	sys 0.043s
I20260709 13:16:30.336447  9772 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.338729  9587 leader_election.cc:304] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:30.340029  9819 raft_consensus.cc:2804] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.340575  9819 raft_consensus.cc:493] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.340322  9813 raft_consensus.cc:359] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.341173  9819 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.341588  9813 raft_consensus.cc:385] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.342058  9813 raft_consensus.cc:740] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.350080  9814 tablet_bootstrap.cc:654] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.343065  9813 consensus_queue.cc:260] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.353683  9819 raft_consensus.cc:515] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.357908  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
W20260709 13:16:30.360051  9587 leader_election.cc:343] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.361368  9813 ts_tablet_manager.cc:1434] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:16:30.366485  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.367105  9772 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.370558  9520 catalog_manager.cc:5697] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.375705  9772 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.376632  9813 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.377476  9587 leader_election.cc:304] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:30.381178  9812 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.381982  9812 ts_tablet_manager.cc:1403] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.028s
I20260709 13:16:30.383453  9838 raft_consensus.cc:2804] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.386569  9812 raft_consensus.cc:359] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.389627  9812 raft_consensus.cc:385] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.389101  9813 tablet_bootstrap.cc:654] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.387697  9819 leader_election.cc:290] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.391813  9814 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.391824  9812 raft_consensus.cc:740] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.394320  9814 ts_tablet_manager.cc:1403] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.063s	user 0.032s	sys 0.015s
I20260709 13:16:30.394814  9812 consensus_queue.cc:260] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.397634  9812 ts_tablet_manager.cc:1434] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 13:16:30.399055  9812 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.399555  9838 raft_consensus.cc:697] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:30.398593  9814 raft_consensus.cc:359] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.401032  9814 raft_consensus.cc:385] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.401414  9814 raft_consensus.cc:740] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.401264  9838 consensus_queue.cc:237] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.402603  9814 consensus_queue.cc:260] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.405407  9814 ts_tablet_manager.cc:1434] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:16:30.408273  9814 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.408797  9812 tablet_bootstrap.cc:654] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:30.419749  9569 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:30.418993  9519 catalog_manager.cc:5697] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.420248  9814 tablet_bootstrap.cc:654] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.423269  9813 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.425462  9813 ts_tablet_manager.cc:1403] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.015s
I20260709 13:16:30.428910  9813 raft_consensus.cc:359] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.432209  9813 raft_consensus.cc:385] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.432695  9813 raft_consensus.cc:740] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.432823  9812 tablet_bootstrap.cc:492] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.433756  9812 ts_tablet_manager.cc:1403] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.020s
I20260709 13:16:30.433688  9813 consensus_queue.cc:260] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.438299  9812 raft_consensus.cc:359] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.439517  9812 raft_consensus.cc:385] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.440021  9812 raft_consensus.cc:740] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.441290  9812 consensus_queue.cc:260] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.443228  9813 ts_tablet_manager.cc:1434] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.005s
I20260709 13:16:30.447531  9812 ts_tablet_manager.cc:1434] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:16:30.449193  9812 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.451030  9813 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.458289  9812 tablet_bootstrap.cc:654] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.458520  9813 tablet_bootstrap.cc:654] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.461691  9814 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.462409  9814 ts_tablet_manager.cc:1403] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.054s	user 0.029s	sys 0.012s
I20260709 13:16:30.463590  9813 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.464115  9813 ts_tablet_manager.cc:1403] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.003s
I20260709 13:16:30.466329  9813 raft_consensus.cc:359] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.466910  9813 raft_consensus.cc:385] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.466414  9814 raft_consensus.cc:359] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.467294  9813 raft_consensus.cc:740] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.467526  9814 raft_consensus.cc:385] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.467979  9814 raft_consensus.cc:740] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.468248  9813 consensus_queue.cc:260] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.468746  9814 consensus_queue.cc:260] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.469584  9812 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.470118  9812 ts_tablet_manager.cc:1403] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.001s
I20260709 13:16:30.470520  9813 ts_tablet_manager.cc:1434] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:30.471978  9813 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.472357  9832 raft_consensus.cc:493] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.472888  9832 raft_consensus.cc:515] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.475033  9832 leader_election.cc:290] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.476125  9821 raft_consensus.cc:493] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.476765  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.476686  9821 raft_consensus.cc:515] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.477506  9623 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:30.478044  9812 raft_consensus.cc:359] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.478987  9738 leader_election.cc:304] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:30.479637  9821 leader_election.cc:290] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.480031  9832 raft_consensus.cc:2804] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.480523  9832 raft_consensus.cc:493] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.480948  9832 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.479199  9813 tablet_bootstrap.cc:654] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.476136  9814 ts_tablet_manager.cc:1434] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 13:16:30.489632  9814 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.493918  9832 raft_consensus.cc:515] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.496748  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.497187  9814 tablet_bootstrap.cc:654] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.497617  9697 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:30.479416  9812 raft_consensus.cc:385] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.499238  9812 raft_consensus.cc:740] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.499519  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:30.500164  9623 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.500247  9812 consensus_queue.cc:260] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.503437  9812 ts_tablet_manager.cc:1434] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:16:30.504737  9812 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.506827  9832 leader_election.cc:290] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.506886  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:30.508087  9697 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.509784  9813 tablet_bootstrap.cc:492] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.510454  9623 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:30.511935  9738 leader_election.cc:304] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:30.512874  9832 raft_consensus.cc:2804] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.510490  9813 ts_tablet_manager.cc:1403] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.007s
I20260709 13:16:30.515000  9832 raft_consensus.cc:697] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:30.516110  9832 consensus_queue.cc:237] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.517856  9812 tablet_bootstrap.cc:654] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.520483  9813 raft_consensus.cc:359] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.521483  9813 raft_consensus.cc:385] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.521898  9813 raft_consensus.cc:740] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.523731  9697 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:30.536067  9812 tablet_bootstrap.cc:492] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.536724  9812 ts_tablet_manager.cc:1403] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.002s
I20260709 13:16:30.538264  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43319ce311c54054990dde238848975d" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.523236  9813 consensus_queue.cc:260] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.540208  9814 tablet_bootstrap.cc:492] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
W20260709 13:16:30.540428  9662 leader_election.cc:343] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.540848  9814 ts_tablet_manager.cc:1403] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.023s
I20260709 13:16:30.540550  9812 raft_consensus.cc:359] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.541458  9812 raft_consensus.cc:385] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.541754  9812 raft_consensus.cc:740] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.541903  9813 ts_tablet_manager.cc:1434] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:16:30.542546  9812 consensus_queue.cc:260] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.544368  9814 raft_consensus.cc:359] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.545495  9812 ts_tablet_manager.cc:1434] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:30.547287  9814 raft_consensus.cc:385] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.547729  9814 raft_consensus.cc:740] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.548135  9812 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:30.548805  9814 consensus_queue.cc:260] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.550640  9520 catalog_manager.cc:5697] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.552098  9814 ts_tablet_manager.cc:1434] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:16:30.553449  9814 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.555353  9812 tablet_bootstrap.cc:654] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.556474  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43319ce311c54054990dde238848975d" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.557325  9623 raft_consensus.cc:2468] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:30.545878  9813 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.559470  9664 leader_election.cc:304] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507
I20260709 13:16:30.560995  9821 raft_consensus.cc:2804] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.561553  9821 raft_consensus.cc:493] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.561978  9821 raft_consensus.cc:3060] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.565781  9812 tablet_bootstrap.cc:492] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:30.566412  9812 ts_tablet_manager.cc:1403] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:16:30.568286  9813 tablet_bootstrap.cc:654] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.561064  9814 tablet_bootstrap.cc:654] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.569795  9812 raft_consensus.cc:359] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.570626  9812 raft_consensus.cc:385] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.570966  9812 raft_consensus.cc:740] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.570878  9821 raft_consensus.cc:515] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.572125  9812 consensus_queue.cc:260] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.574486  9812 ts_tablet_manager.cc:1434] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:30.574563  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43319ce311c54054990dde238848975d" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:30.575322  9623 raft_consensus.cc:3060] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.581130  9814 tablet_bootstrap.cc:492] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.581715  9814 ts_tablet_manager.cc:1403] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.001s
I20260709 13:16:30.584199  9821 leader_election.cc:290] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.584969  9623 raft_consensus.cc:2468] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:30.586040  9813 tablet_bootstrap.cc:492] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.586601  9813 ts_tablet_manager.cc:1403] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.041s	user 0.005s	sys 0.008s
I20260709 13:16:30.586445  9664 leader_election.cc:304] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.587136  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43319ce311c54054990dde238848975d" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.587563  9821 raft_consensus.cc:2804] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.587796  9772 raft_consensus.cc:3060] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.589030  9814 raft_consensus.cc:359] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.589797  9814 raft_consensus.cc:385] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.590080  9814 raft_consensus.cc:740] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.590646  9813 raft_consensus.cc:359] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.591027  9814 consensus_queue.cc:260] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.591888  9813 raft_consensus.cc:385] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.592458  9813 raft_consensus.cc:740] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.593524  9814 ts_tablet_manager.cc:1434] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:16:30.593575  9813 consensus_queue.cc:260] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.594859  9814 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:30.586609  9832 raft_consensus.cc:493] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.595769  9832 raft_consensus.cc:515] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.599925  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.600304  9832 leader_election.cc:290] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.600804  9697 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:30.601269  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.600008  9772 raft_consensus.cc:2468] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:30.602511  9736 leader_election.cc:304] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.602912  9623 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:30.602314  9814 tablet_bootstrap.cc:654] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.603428  9821 raft_consensus.cc:697] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:30.604655  9821 consensus_queue.cc:237] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.608209  9832 raft_consensus.cc:2804] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.612100  9832 raft_consensus.cc:493] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.614094  9832 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.612425  9813 ts_tablet_manager.cc:1434] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.010s
I20260709 13:16:30.617633  9813 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.620505  9519 catalog_manager.cc:5697] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.626483  9813 tablet_bootstrap.cc:654] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.627036  9832 raft_consensus.cc:515] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.630312  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:30.630990  9623 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.633610  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:30.634274  9697 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.637203  9832 leader_election.cc:290] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:30.630419  9814 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:30.639048  9814 ts_tablet_manager.cc:1403] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.009s
I20260709 13:16:30.639153  9623 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:30.640583  9738 leader_election.cc:304] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:30.641045  9813 tablet_bootstrap.cc:492] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.641502  9832 raft_consensus.cc:2804] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.642268  9832 raft_consensus.cc:697] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:30.642246  9814 raft_consensus.cc:359] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.643079  9821 raft_consensus.cc:493] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.643232  9814 raft_consensus.cc:385] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.643707  9814 raft_consensus.cc:740] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.643435  9832 consensus_queue.cc:237] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:30.643728  9821 raft_consensus.cc:515] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.644829  9814 consensus_queue.cc:260] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.641587  9813 ts_tablet_manager.cc:1403] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.015s
I20260709 13:16:30.647706  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.648425  9622 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:30.649067  9838 raft_consensus.cc:493] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.649617  9838 raft_consensus.cc:515] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.650197  9664 leader_election.cc:304] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.650629  9813 raft_consensus.cc:359] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.651989  9838 leader_election.cc:290] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.652199  9858 raft_consensus.cc:2804] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.652572  9814 ts_tablet_manager.cc:1434] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:30.651448  9813 raft_consensus.cc:385] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.654680  9813 raft_consensus.cc:740] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.654778  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.655654  9698 raft_consensus.cc:2468] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.656126  9813 consensus_queue.cc:260] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.657018  9857 raft_consensus.cc:493] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.656922  9587 leader_election.cc:304] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.657677  9857 raft_consensus.cc:515] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.654083  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.660838  9772 raft_consensus.cc:2468] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.661420  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.661751  9838 raft_consensus.cc:2804] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.662326  9838 raft_consensus.cc:493] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.662734  9838 raft_consensus.cc:3060] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.663273  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.662475  9771 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:30.652916  9858 raft_consensus.cc:493] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.666765  9858 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.668006  9520 catalog_manager.cc:5697] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.671759  9838 raft_consensus.cc:515] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.673194  9623 raft_consensus.cc:2393] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1e5cfbaa43444d1b773156f512e2d11 in current term 1: Already voted for candidate bdf041618a254a1a9d5a2f014162cf90 in this term.
I20260709 13:16:30.674310  9838 leader_election.cc:290] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.672076  9857 leader_election.cc:290] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.676285  9813 ts_tablet_manager.cc:1434] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.029s	user 0.017s	sys 0.007s
I20260709 13:16:30.677368  9813 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:30.679436  9838 raft_consensus.cc:493] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.679818  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:30.680063  9838 raft_consensus.cc:515] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.680512  9698 raft_consensus.cc:3060] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.682444  9838 leader_election.cc:290] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.684146  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.684993  9771 raft_consensus.cc:2468] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:30.684139  9813 tablet_bootstrap.cc:654] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.685845  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc8fd062d764891b4df49bafb99de16" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.686513  9772 raft_consensus.cc:3060] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.688305  9696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.689071  9696 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.689172  9821 leader_election.cc:290] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.691819  9587 leader_election.cc:304] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.695701  9858 raft_consensus.cc:515] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.696969  9662 leader_election.cc:304] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:30.698370  9858 leader_election.cc:290] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.650333  9697 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:30.699293  9838 raft_consensus.cc:2804] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.699308  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:30.700177  9838 raft_consensus.cc:493] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.700189  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.700769  9770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.700928  9838 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.701156  9771 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.701509  9770 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.700316  9623 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.709338  9838 raft_consensus.cc:515] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.710968  9698 raft_consensus.cc:2468] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.711640  9858 raft_consensus.cc:2764] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:16:30.714654  9838 leader_election.cc:290] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.714922  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.717700  9771 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.717777  9587 leader_election.cc:304] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.719694  9772 raft_consensus.cc:2468] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.720575  9770 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:30.716495  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:30.721407  9838 raft_consensus.cc:2804] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.722047  9697 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.722939  9662 leader_election.cc:304] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.722199  9838 raft_consensus.cc:697] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:30.716707  9623 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:30.724022  9858 raft_consensus.cc:2804] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.726912  9858 raft_consensus.cc:697] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:30.727485  9838 consensus_queue.cc:237] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.727783  9858 consensus_queue.cc:237] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.737967  9771 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.738478  9813 tablet_bootstrap.cc:492] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:30.739059  9813 ts_tablet_manager.cc:1403] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.030s
I20260709 13:16:30.739807  9587 leader_election.cc:304] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:30.741021  9819 raft_consensus.cc:2804] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.741623  9819 raft_consensus.cc:697] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:30.742486  9813 raft_consensus.cc:359] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.743371  9813 raft_consensus.cc:385] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.743698  9813 raft_consensus.cc:740] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.744418  9813 consensus_queue.cc:260] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.745723  9697 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.746987  9813 ts_tablet_manager.cc:1434] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.008s
I20260709 13:16:30.747334  9698 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.742552  9819 consensus_queue.cc:237] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.752146  9519 catalog_manager.cc:5697] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.758554  9832 consensus_queue.cc:1048] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:30.760022  9520 catalog_manager.cc:5697] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.781172  9818 consensus_queue.cc:1048] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.786695  9520 catalog_manager.cc:5697] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.872819  9838 raft_consensus.cc:493] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.873409  9838 raft_consensus.cc:515] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.875106  9838 leader_election.cc:290] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.876036  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:30.876248  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.876627  9698 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.876909  9771 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:30.877681  9587 leader_election.cc:304] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.878444  9838 raft_consensus.cc:2804] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.878763  9838 raft_consensus.cc:493] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.879078  9838 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.884297  9838 raft_consensus.cc:515] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.885932  9838 leader_election.cc:290] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.886842  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:30.887190  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:30.887544  9698 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.887800  9771 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.894835  9771 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.894835  9698 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:30.896198  9587 leader_election.cc:304] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:30.896909  9838 raft_consensus.cc:2804] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.897413  9838 raft_consensus.cc:697] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:30.898193  9838 consensus_queue.cc:237] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.905470  9520 catalog_manager.cc:5697] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.955936  9858 raft_consensus.cc:493] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.956449  9858 raft_consensus.cc:515] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:30.958165  9858 leader_election.cc:290] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:30.959139  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:30.959265  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:30.960204  9771 raft_consensus.cc:2393] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1e5cfbaa43444d1b773156f512e2d11 in current term 1: Already voted for candidate bdf041618a254a1a9d5a2f014162cf90 in this term.
I20260709 13:16:30.961287  9662 leader_election.cc:304] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:30.961913  9858 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.967149  9858 raft_consensus.cc:2749] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.972476  9838 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.985736  9839 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:30.986104  9818 consensus_queue.cc:1048] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.004863  9878 consensus_queue.cc:1048] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:31.028059  9771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:31.028661  9771 raft_consensus.cc:606] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:31.033989  9771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:31.034519  9771 raft_consensus.cc:606] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:31.051424  9698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
mode: GRACEFUL
new_leader_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:49028
I20260709 13:16:31.052086  9698 raft_consensus.cc:606] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Received request to transfer leadership to TS bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:31.060284  9771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:31.060902  9771 raft_consensus.cc:606] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:31.074018  9623 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:45296
I20260709 13:16:31.074563  9623 raft_consensus.cc:606] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:31.079851  9623 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:45296
I20260709 13:16:31.080441  9623 raft_consensus.cc:606] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:31.085732  9623 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:45296
I20260709 13:16:31.086246  9623 raft_consensus.cc:606] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:31.104739  9878 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.109019  9858 consensus_queue.cc:1048] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.109776  9839 consensus_queue.cc:1048] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:31.117776  9818 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.130642  9819 consensus_queue.cc:1048] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.135475  9857 consensus_queue.cc:1048] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.138306  9818 raft_consensus.cc:993] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: : Instructing follower bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.144908  9877 raft_consensus.cc:1081] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.148283  9622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:45290
I20260709 13:16:31.148871  9622 raft_consensus.cc:493] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.149204  9622 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.157094  9622 raft_consensus.cc:515] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:31.158936  9622 leader_election.cc:290] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Requested vote from peers ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:31.159806  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.160475  9772 raft_consensus.cc:3055] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.160557  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:31.160844  9772 raft_consensus.cc:740] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:31.161235  9698 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.161775  9772 consensus_queue.cc:260] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:31.163130  9772 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.168467  9698 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.169693  9587 leader_election.cc:304] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.170090  9772 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.170469  9819 raft_consensus.cc:2804] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.171000  9819 raft_consensus.cc:697] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.171850  9819 consensus_queue.cc:237] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:31.178013  9520 catalog_manager.cc:5697] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.209810  9819 consensus_queue.cc:1048] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.220777  9819 consensus_queue.cc:1048] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.230890  9819 raft_consensus.cc:993] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.231415  9819 raft_consensus.cc:1081] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.233026  9698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49010
I20260709 13:16:31.233669  9698 raft_consensus.cc:493] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.234053  9698 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.239827  9698 raft_consensus.cc:515] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.241348  9698 leader_election.cc:290] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:31.242170  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:31.242394  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3baf9fcdf364ef0b6d9fbe7e4b22dfc" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.243046  9772 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.243046  9622 raft_consensus.cc:3055] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.243571  9622 raft_consensus.cc:740] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.244387  9622 consensus_queue.cc:260] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.245623  9622 raft_consensus.cc:3060] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.250337  9772 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.251533  9662 leader_election.cc:304] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.252415  9857 raft_consensus.cc:2804] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.252775  9622 raft_consensus.cc:2468] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.252888  9857 raft_consensus.cc:697] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:31.253922  9857 consensus_queue.cc:237] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.260043  9520 catalog_manager.cc:5697] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.264357  9818 raft_consensus.cc:993] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: : Instructing follower bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.264873  9818 raft_consensus.cc:1081] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.266439  9622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:45290
I20260709 13:16:31.266917  9622 raft_consensus.cc:493] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.267234  9622 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.272704  9622 raft_consensus.cc:515] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.274266  9622 leader_election.cc:290] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:31.275197  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:31.275514  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40a69de06904e52a6642505b5a42db1" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.275846  9698 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.276136  9772 raft_consensus.cc:3055] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.276482  9772 raft_consensus.cc:740] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:31.277175  9772 consensus_queue.cc:260] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.278141  9772 raft_consensus.cc:3060] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.282306  9698 raft_consensus.cc:2468] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.283416  9772 raft_consensus.cc:2468] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.283538  9587 leader_election.cc:304] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.284618  9819 raft_consensus.cc:2804] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.285125  9819 raft_consensus.cc:697] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.285768  9819 consensus_queue.cc:237] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.292496  9520 catalog_manager.cc:5697] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.295522  9857 consensus_queue.cc:1048] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.308530  9821 consensus_queue.cc:1048] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.316166  9857 raft_consensus.cc:993] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: : Instructing follower bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.316748  9821 raft_consensus.cc:1081] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Signalling peer bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:31.318267  9622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:45292
I20260709 13:16:31.318770  9622 raft_consensus.cc:493] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.319108  9622 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.324345  9622 raft_consensus.cc:515] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.326128  9622 leader_election.cc:290] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:31.327375  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.327800  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96316755af8c4c9dac15c7556b76e4cc" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:31.328167  9772 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.328500  9698 raft_consensus.cc:3055] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.328864  9698 raft_consensus.cc:740] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:31.329807  9698 consensus_queue.cc:260] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.331241  9698 raft_consensus.cc:3060] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.336289  9772 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.337638  9587 leader_election.cc:304] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:31.338379  9819 raft_consensus.cc:2804] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.338382  9698 raft_consensus.cc:2468] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:31.338953  9819 raft_consensus.cc:697] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.339835  9819 consensus_queue.cc:237] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.346243  9518 catalog_manager.cc:5697] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 1 to 2, leader changed from c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2) to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.380185  9819 consensus_queue.cc:1048] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.391266  9838 consensus_queue.cc:1048] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:31.399719  9894 raft_consensus.cc:993] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:31.400333  9838 raft_consensus.cc:1081] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:31.402091  9772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50270
I20260709 13:16:31.402612  9772 raft_consensus.cc:493] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.403036  9772 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.409729  9772 raft_consensus.cc:515] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.412654  9772 leader_election.cc:290] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:31.412964  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:31.413400  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1787d4051c14afcac84c33f9d4ff691" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:31.413751  9622 raft_consensus.cc:3055] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.414089  9698 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.414117  9622 raft_consensus.cc:740] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.415295  9622 consensus_queue.cc:260] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.416888  9622 raft_consensus.cc:3060] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.422290  9698 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:31.423655  9736 leader_election.cc:304] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.424587  9818 raft_consensus.cc:2804] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.424583  9622 raft_consensus.cc:2468] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:31.425387  9818 raft_consensus.cc:697] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:31.426330  9818 consensus_queue.cc:237] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.433938  9520 catalog_manager.cc:5697] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 1 to 2, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.439452  9894 raft_consensus.cc:993] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.440140  9838 raft_consensus.cc:1081] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.442194  9698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49010
I20260709 13:16:31.443670  9698 raft_consensus.cc:493] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.444121  9698 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.444716  9878 raft_consensus.cc:993] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.445302  9818 raft_consensus.cc:1081] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:31.446976  9697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49024
I20260709 13:16:31.447574  9697 raft_consensus.cc:493] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:31.448017  9697 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.450338  9698 raft_consensus.cc:515] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.452590  9698 leader_election.cc:290] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:31.453768  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.454061  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:31.454516  9772 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.454679  9622 raft_consensus.cc:3055] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.455163  9622 raft_consensus.cc:740] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:31.455757  9697 raft_consensus.cc:515] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.455925  9622 consensus_queue.cc:260] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.457177  9622 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.459038  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:31.459506  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a4299f51a58415a93d06cf2685ea63a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:31.460225  9771 raft_consensus.cc:3055] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:31.460600  9771 raft_consensus.cc:740] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:31.461506  9771 consensus_queue.cc:260] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.462594  9772 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.462738  9771 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.459698  9623 raft_consensus.cc:3060] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:31.463335  9622 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.464409  9664 leader_election.cc:304] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.465257  9821 raft_consensus.cc:2804] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.465622  9697 leader_election.cc:290] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:31.465740  9821 raft_consensus.cc:697] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:31.466784  9821 consensus_queue.cc:237] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.470687  9623 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.471529  9771 raft_consensus.cc:2468] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:31.472167  9664 leader_election.cc:304] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:31.473080  9821 raft_consensus.cc:2804] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:31.473529  9821 raft_consensus.cc:697] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:31.474447  9821 consensus_queue.cc:237] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:31.476038  9520 catalog_manager.cc:5697] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.488431  9520 catalog_manager.cc:5697] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:31.591321  9697 raft_consensus.cc:1275] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.592653  9894 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.601425  9771 raft_consensus.cc:1275] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.602938  9894 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.646878  9771 raft_consensus.cc:1275] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.648113  9821 consensus_queue.cc:1048] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.655754  9623 raft_consensus.cc:1275] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.657649  9858 consensus_queue.cc:1048] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.709157  9771 raft_consensus.cc:1275] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.710556  9894 consensus_queue.cc:1048] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.720377  9697 raft_consensus.cc:1275] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.722010  9838 consensus_queue.cc:1048] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.775164  9697 raft_consensus.cc:1275] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.776688  9838 consensus_queue.cc:1048] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.785822  9771 raft_consensus.cc:1275] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.787474  9838 consensus_queue.cc:1048] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.904707  9623 raft_consensus.cc:1275] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.906102  9818 consensus_queue.cc:1048] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.914884  9697 raft_consensus.cc:1275] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.916473  9877 consensus_queue.cc:1048] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.969168  9623 raft_consensus.cc:1275] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.970538  9858 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.978550  9771 raft_consensus.cc:1275] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.980293  9857 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:31.998474  9623 raft_consensus.cc:1275] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:31.999787  9857 consensus_queue.cc:1048] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.009236  9771 raft_consensus.cc:1275] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:32.011554  9821 consensus_queue.cc:1048] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.121522  9518 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58680:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:34.124733  9518 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:34.172562  9605 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed743650b834413a19d90748f427f58 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:34.173771  9605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed743650b834413a19d90748f427f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.175266  9680 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed743650b834413a19d90748f427f58 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:34.176553  9680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed743650b834413a19d90748f427f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.176250  9760 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed743650b834413a19d90748f427f58 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:34.177700  9760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed743650b834413a19d90748f427f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.179401  9613 tablet_service.cc:1511] Processing CreateTablet for tablet 881ffabead614781aee7016272cd256f (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:34.180683  9613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881ffabead614781aee7016272cd256f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.184526  9681 tablet_service.cc:1511] Processing CreateTablet for tablet 881ffabead614781aee7016272cd256f (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:34.185549  9606 tablet_service.cc:1511] Processing CreateTablet for tablet d016eb1f6f184f21be6a986e4086b5ec (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:34.186812  9758 tablet_service.cc:1511] Processing CreateTablet for tablet 881ffabead614781aee7016272cd256f (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:34.188077  9758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881ffabead614781aee7016272cd256f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.188952  9687 tablet_service.cc:1511] Processing CreateTablet for tablet d016eb1f6f184f21be6a986e4086b5ec (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:34.191012  9757 tablet_service.cc:1511] Processing CreateTablet for tablet d016eb1f6f184f21be6a986e4086b5ec (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:34.192333  9757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d016eb1f6f184f21be6a986e4086b5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.193877  9682 tablet_service.cc:1511] Processing CreateTablet for tablet 6f60df8c06c94fd5a561bc5497790e72 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:34.193991  9754 tablet_service.cc:1511] Processing CreateTablet for tablet 6f60df8c06c94fd5a561bc5497790e72 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:34.195883  9754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f60df8c06c94fd5a561bc5497790e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.198004  9688 tablet_service.cc:1511] Processing CreateTablet for tablet 39d1254e35c848718bd9ff4c211bdd9c (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:34.199725  9762 tablet_service.cc:1511] Processing CreateTablet for tablet 39d1254e35c848718bd9ff4c211bdd9c (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:34.198007  9610 tablet_service.cc:1511] Processing CreateTablet for tablet 6f60df8c06c94fd5a561bc5497790e72 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:34.202127  9607 tablet_service.cc:1511] Processing CreateTablet for tablet 39d1254e35c848718bd9ff4c211bdd9c (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:34.201041  9762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d1254e35c848718bd9ff4c211bdd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.186853  9606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d016eb1f6f184f21be6a986e4086b5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.204156  9612 tablet_service.cc:1511] Processing CreateTablet for tablet ecf55527ffda46a3b33fe2c23a24d553 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:34.205387  9612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf55527ffda46a3b33fe2c23a24d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.206350  9610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f60df8c06c94fd5a561bc5497790e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.199728  9681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881ffabead614781aee7016272cd256f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.207890  9682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f60df8c06c94fd5a561bc5497790e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.208279  9607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d1254e35c848718bd9ff4c211bdd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.210150  9687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d016eb1f6f184f21be6a986e4086b5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.213184  9688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d1254e35c848718bd9ff4c211bdd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.243216  9684 tablet_service.cc:1511] Processing CreateTablet for tablet ecf55527ffda46a3b33fe2c23a24d553 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:34.245126  9684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf55527ffda46a3b33fe2c23a24d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.248337  9685 tablet_service.cc:1511] Processing CreateTablet for tablet 0371001f236f47a397a0e216a7abd61a (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:34.249712  9685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371001f236f47a397a0e216a7abd61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.249502  9608 tablet_service.cc:1511] Processing CreateTablet for tablet 0371001f236f47a397a0e216a7abd61a (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:34.250772  9608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371001f236f47a397a0e216a7abd61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.256899  9759 tablet_service.cc:1511] Processing CreateTablet for tablet ecf55527ffda46a3b33fe2c23a24d553 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:34.259501  9759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf55527ffda46a3b33fe2c23a24d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.261296  9756 tablet_service.cc:1511] Processing CreateTablet for tablet 0371001f236f47a397a0e216a7abd61a (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:34.262776  9756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371001f236f47a397a0e216a7abd61a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.268597  9609 tablet_service.cc:1511] Processing CreateTablet for tablet 54d3de9f4fe949d79e675ef0630c1398 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:34.270136  9609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54d3de9f4fe949d79e675ef0630c1398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.274622  9683 tablet_service.cc:1511] Processing CreateTablet for tablet 54d3de9f4fe949d79e675ef0630c1398 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:34.275960  9683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54d3de9f4fe949d79e675ef0630c1398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.292205  9913 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.299147  9760 tablet_service.cc:1511] Processing CreateTablet for tablet 54d3de9f4fe949d79e675ef0630c1398 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:34.300567  9760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54d3de9f4fe949d79e675ef0630c1398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.303040  9611 tablet_service.cc:1511] Processing CreateTablet for tablet 16e27fb303e045ae8e7cc648bacfdd77 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:34.304378  9611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e27fb303e045ae8e7cc648bacfdd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.311447  9686 tablet_service.cc:1511] Processing CreateTablet for tablet 16e27fb303e045ae8e7cc648bacfdd77 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:34.314618  9686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e27fb303e045ae8e7cc648bacfdd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.321646  9761 tablet_service.cc:1511] Processing CreateTablet for tablet 16e27fb303e045ae8e7cc648bacfdd77 (DEFAULT_TABLE table=soft_deleted_table [id=840a67c9ba24414d9a754136c4472599]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:34.323114  9761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e27fb303e045ae8e7cc648bacfdd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:34.343254  9913 tablet_bootstrap.cc:654] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.360251  9915 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.371591  9915 tablet_bootstrap.cc:654] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.392516  9914 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.397502  9913 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.399293  9913 ts_tablet_manager.cc:1403] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.000s
I20260709 13:16:34.400391  9914 tablet_bootstrap.cc:654] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.407727  9913 raft_consensus.cc:359] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.410074  9913 raft_consensus.cc:385] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.410391  9913 raft_consensus.cc:740] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.411463  9913 consensus_queue.cc:260] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.412925  9915 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.413640  9914 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.414309  9915 ts_tablet_manager.cc:1403] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.006s
I20260709 13:16:34.414464  9914 ts_tablet_manager.cc:1403] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.011s
I20260709 13:16:34.418015  9914 raft_consensus.cc:359] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.418823  9914 raft_consensus.cc:385] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.419005  9913 ts_tablet_manager.cc:1434] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:16:34.419176  9914 raft_consensus.cc:740] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.420173  9913 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.420017  9914 consensus_queue.cc:260] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.421149  9915 raft_consensus.cc:359] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.421958  9915 raft_consensus.cc:385] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.422281  9915 raft_consensus.cc:740] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.422389  9914 ts_tablet_manager.cc:1434] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:34.423592  9914 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.423234  9915 consensus_queue.cc:260] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.429946  9913 tablet_bootstrap.cc:654] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.430727  9914 tablet_bootstrap.cc:654] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.436532  9915 ts_tablet_manager.cc:1434] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:16:34.437587  9915 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.437616  9914 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.438313  9914 ts_tablet_manager.cc:1403] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.005s
I20260709 13:16:34.439328  9913 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.439934  9913 ts_tablet_manager.cc:1403] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.002s
I20260709 13:16:34.441341  9914 raft_consensus.cc:359] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.442152  9914 raft_consensus.cc:385] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.442502  9914 raft_consensus.cc:740] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.443095  9913 raft_consensus.cc:359] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.443264  9914 consensus_queue.cc:260] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.443986  9913 raft_consensus.cc:385] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.444408  9913 raft_consensus.cc:740] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.445151  9915 tablet_bootstrap.cc:654] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.445788  9914 ts_tablet_manager.cc:1434] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:34.445336  9913 consensus_queue.cc:260] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.447027  9914 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.451176  9913 ts_tablet_manager.cc:1434] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:16:34.457767  9914 tablet_bootstrap.cc:654] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:34.461491  9569 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:34.467176  9913 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.467532  9914 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.468173  9914 ts_tablet_manager.cc:1403] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.007s
I20260709 13:16:34.469264  9915 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.469817  9915 ts_tablet_manager.cc:1403] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.003s
I20260709 13:16:34.470866  9914 raft_consensus.cc:359] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.471508  9914 raft_consensus.cc:385] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.471829  9914 raft_consensus.cc:740] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.472683  9914 consensus_queue.cc:260] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.474378  9914 ts_tablet_manager.cc:1434] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:34.474536  9913 tablet_bootstrap.cc:654] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.475373  9914 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.479671  9915 raft_consensus.cc:359] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.480723  9915 raft_consensus.cc:385] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.481118  9915 raft_consensus.cc:740] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.482064  9914 tablet_bootstrap.cc:654] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.482091  9915 consensus_queue.cc:260] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.485193  9915 ts_tablet_manager.cc:1434] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:16:34.486207  9915 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.486370  9913 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.486977  9913 ts_tablet_manager.cc:1403] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:16:34.490295  9913 raft_consensus.cc:359] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.491261  9913 raft_consensus.cc:385] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.491659  9913 raft_consensus.cc:740] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.492591  9913 consensus_queue.cc:260] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.493880  9915 tablet_bootstrap.cc:654] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.505564  9913 ts_tablet_manager.cc:1434] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:16:34.516650  9914 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.517335  9914 ts_tablet_manager.cc:1403] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.023s
I20260709 13:16:34.519469  9913 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.519681  9914 raft_consensus.cc:359] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.520639  9914 raft_consensus.cc:385] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.520954  9914 raft_consensus.cc:740] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.521633  9914 consensus_queue.cc:260] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.522320  9915 tablet_bootstrap.cc:492] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.522953  9915 ts_tablet_manager.cc:1403] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.017s
I20260709 13:16:34.523910  9914 ts_tablet_manager.cc:1434] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:34.524945  9914 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.526243  9915 raft_consensus.cc:359] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.527024  9913 tablet_bootstrap.cc:654] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.527086  9915 raft_consensus.cc:385] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.527812  9915 raft_consensus.cc:740] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.528677  9915 consensus_queue.cc:260] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.532012  9914 tablet_bootstrap.cc:654] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.534682  9915 ts_tablet_manager.cc:1434] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.006s
I20260709 13:16:34.536294  9915 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.546734  9915 tablet_bootstrap.cc:654] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.549558  9914 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.549790  9913 tablet_bootstrap.cc:492] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.550246  9914 ts_tablet_manager.cc:1403] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.012s
I20260709 13:16:34.550586  9913 ts_tablet_manager.cc:1403] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.009s
I20260709 13:16:34.554375  9914 raft_consensus.cc:359] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.554863  9913 raft_consensus.cc:359] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.555433  9914 raft_consensus.cc:385] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.555778  9913 raft_consensus.cc:385] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.555999  9914 raft_consensus.cc:740] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.556234  9915 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.556222  9913 raft_consensus.cc:740] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.557037  9915 ts_tablet_manager.cc:1403] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.002s
I20260709 13:16:34.556938  9914 consensus_queue.cc:260] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.557507  9913 consensus_queue.cc:260] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.559393  9914 ts_tablet_manager.cc:1434] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:16:34.560479  9914 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.560217  9915 raft_consensus.cc:359] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.561028  9915 raft_consensus.cc:385] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.562219  9915 raft_consensus.cc:740] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.562992  9915 consensus_queue.cc:260] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.566999  9913 ts_tablet_manager.cc:1434] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.008s
I20260709 13:16:34.567714  9914 tablet_bootstrap.cc:654] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.569959  9915 ts_tablet_manager.cc:1434] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:16:34.568578  9901 raft_consensus.cc:493] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.576864  9915 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.576637  9901 raft_consensus.cc:515] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.584303  9915 tablet_bootstrap.cc:654] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.584184  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:34.585215  9623 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:34.587132  9738 leader_election.cc:304] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:34.595146  9901 leader_election.cc:290] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.596516  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.596837  9901 raft_consensus.cc:2804] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.597240  9901 raft_consensus.cc:493] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.597318  9697 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:34.597657  9901 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.568173  9913 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.616627  9915 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.617383  9915 ts_tablet_manager.cc:1403] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.014s
I20260709 13:16:34.620313  9915 raft_consensus.cc:359] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.621089  9915 raft_consensus.cc:385] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.621513  9915 raft_consensus.cc:740] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.625572  9915 consensus_queue.cc:260] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.628194  9915 ts_tablet_manager.cc:1434] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:16:34.629302  9915 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.628600  9931 raft_consensus.cc:493] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.637404  9915 tablet_bootstrap.cc:654] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.637197  9931 raft_consensus.cc:515] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.641032  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
W20260709 13:16:34.643021  9738 leader_election.cc:343] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.645581  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.646548  9697 raft_consensus.cc:2468] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:34.647423  9901 raft_consensus.cc:515] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.648186  9931 leader_election.cc:290] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.648257  9736 leader_election.cc:304] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:34.649612  9931 raft_consensus.cc:2804] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.649935  9901 leader_election.cc:290] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.649981  9931 raft_consensus.cc:493] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.650470  9914 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.650636  9931 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.650978  9914 ts_tablet_manager.cc:1403] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.091s	user 0.027s	sys 0.051s
I20260709 13:16:34.651067  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.651752  9697 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.651823  9913 tablet_bootstrap.cc:654] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.653997  9914 raft_consensus.cc:359] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.654721  9914 raft_consensus.cc:385] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.655017  9914 raft_consensus.cc:740] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.655742  9914 consensus_queue.cc:260] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.656742  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:34.657369  9623 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.658203  9914 ts_tablet_manager.cc:1434] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:34.658720  9931 raft_consensus.cc:515] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.659528  9838 raft_consensus.cc:493] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.660048  9838 raft_consensus.cc:515] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.662451  9838 leader_election.cc:290] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.662806  9915 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.663362  9915 ts_tablet_manager.cc:1403] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.008s
I20260709 13:16:34.664757  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "881ffabead614781aee7016272cd256f" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.665432  9698 raft_consensus.cc:2468] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:34.665320  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "881ffabead614781aee7016272cd256f" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:34.666467  9772 raft_consensus.cc:2468] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:34.666939  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:34.668495  9931 leader_election.cc:290] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.667311  9915 raft_consensus.cc:359] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.668767  9587 leader_election.cc:304] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.669446  9915 raft_consensus.cc:385] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.669777  9915 raft_consensus.cc:740] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.670622  9838 raft_consensus.cc:2804] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.670495  9915 consensus_queue.cc:260] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.671157  9838 raft_consensus.cc:493] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.671880  9838 raft_consensus.cc:3060] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.677130  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.659268  9914 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.678010  9698 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:34.683007  9738 leader_election.cc:343] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.685757  9838 raft_consensus.cc:515] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.692413  9696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "881ffabead614781aee7016272cd256f" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.692801  9697 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:34.693006  9696 raft_consensus.cc:3060] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.694221  9736 leader_election.cc:304] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.695073  9931 raft_consensus.cc:2804] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.695380  9623 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:34.695572  9931 raft_consensus.cc:697] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:34.688740  9914 tablet_bootstrap.cc:654] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.696496  9931 consensus_queue.cc:237] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.704324  9935 raft_consensus.cc:493] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.706240  9935 raft_consensus.cc:515] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.707757  9696 raft_consensus.cc:2468] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:34.711383  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:34.706837  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "881ffabead614781aee7016272cd256f" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:34.712563  9772 raft_consensus.cc:3060] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.708293  9913 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.713459  9913 ts_tablet_manager.cc:1403] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.146s	user 0.030s	sys 0.026s
I20260709 13:16:34.716180  9587 leader_election.cc:304] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.716681  9913 raft_consensus.cc:359] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
W20260709 13:16:34.717187  9587 leader_election.cc:343] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.717458  9913 raft_consensus.cc:385] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.717779  9913 raft_consensus.cc:740] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.718544  9913 consensus_queue.cc:260] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.720978  9913 ts_tablet_manager.cc:1434] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:34.721107  9698 raft_consensus.cc:2468] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:34.709833  9935 leader_election.cc:290] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.710335  9915 ts_tablet_manager.cc:1434] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.047s	user 0.012s	sys 0.008s
I20260709 13:16:34.710958  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.722638  9736 leader_election.cc:304] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:34.722043  9913 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
W20260709 13:16:34.723048  9587 leader_election.cc:343] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.708204  9838 leader_election.cc:290] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.722062  9935 raft_consensus.cc:2804] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.723721  9587 leader_election.cc:304] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90; no voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:34.722975  9915 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.725788  9838 raft_consensus.cc:2749] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:34.726624  9838 raft_consensus.cc:493] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.727582  9838 raft_consensus.cc:515] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.724179  9772 raft_consensus.cc:2468] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:34.730528  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.731178  9915 tablet_bootstrap.cc:654] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.731187  9697 raft_consensus.cc:2468] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:34.724818  9935 raft_consensus.cc:697] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:34.730520  9913 tablet_bootstrap.cc:654] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.732471  9935 consensus_queue.cc:237] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.735636  9901 raft_consensus.cc:2804] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.732818  9587 leader_election.cc:304] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.741380  9838 leader_election.cc:290] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.743790  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:34.745652  9838 raft_consensus.cc:2804] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.747202  9838 raft_consensus.cc:493] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.748829  9838 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:34.748135  9587 leader_election.cc:343] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.752949  9901 raft_consensus.cc:697] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:34.754096  9901 consensus_queue.cc:237] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.799470  9913 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.801134  9913 ts_tablet_manager.cc:1403] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.014s
I20260709 13:16:34.799471  9838 raft_consensus.cc:515] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.806882  9914 tablet_bootstrap.cc:492] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.808086  9914 ts_tablet_manager.cc:1403] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.149s	user 0.018s	sys 0.022s
I20260709 13:16:34.806908  9518 catalog_manager.cc:5697] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:34.810662  9913 raft_consensus.cc:359] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.811743  9913 raft_consensus.cc:385] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.812124  9913 raft_consensus.cc:740] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.813153  9913 consensus_queue.cc:260] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.814625  9915 tablet_bootstrap.cc:492] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.815775  9915 ts_tablet_manager.cc:1403] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.007s
I20260709 13:16:34.817876  9913 ts_tablet_manager.cc:1434] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.012s
I20260709 13:16:34.819708  9913 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:34.823033  9838 leader_election.cc:290] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.818614  9519 catalog_manager.cc:5697] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:34.824246  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:34.825299  9915 raft_consensus.cc:359] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.826272  9915 raft_consensus.cc:385] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.825335  9900 raft_consensus.cc:493] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.826845  9942 raft_consensus.cc:493] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.826937  9915 raft_consensus.cc:740] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.827984  9900 raft_consensus.cc:515] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
W20260709 13:16:34.826450  9587 leader_election.cc:343] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.828069  9942 raft_consensus.cc:515] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.828343  9915 consensus_queue.cc:260] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.833094  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:34.824270  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.828550  9914 raft_consensus.cc:359] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.834408  9698 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:34.835563  9664 leader_election.cc:343] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.838769  9900 leader_election.cc:290] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.839332  9915 ts_tablet_manager.cc:1434] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 13:16:34.841601  9915 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.842217  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:34.834534  9914 raft_consensus.cc:385] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.843765  9914 raft_consensus.cc:740] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
W20260709 13:16:34.844108  9662 leader_election.cc:343] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.844740  9662 leader_election.cc:304] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:34.845009  9914 consensus_queue.cc:260] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.846637  9930 raft_consensus.cc:2749] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:34.849952  9913 tablet_bootstrap.cc:654] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.851702  9914 ts_tablet_manager.cc:1434] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.005s
I20260709 13:16:34.853615  9914 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.855705  9838 raft_consensus.cc:493] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.856357  9838 raft_consensus.cc:515] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.862219  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.863200  9697 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:34.865176  9587 leader_election.cc:304] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.866671  9937 raft_consensus.cc:2804] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.867201  9937 raft_consensus.cc:493] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.867786  9937 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.868404  9931 raft_consensus.cc:493] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.868984  9931 raft_consensus.cc:515] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.872454  9931 leader_election.cc:290] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.873996  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
W20260709 13:16:34.875725  9736 leader_election.cc:343] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.878700  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:34.879743  9622 raft_consensus.cc:2468] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:34.880185  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:34.881338  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:34.882303  9838 leader_election.cc:290] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.883040  9930 raft_consensus.cc:493] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.883774  9930 raft_consensus.cc:515] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.885330  9914 tablet_bootstrap.cc:654] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.886430  9930 leader_election.cc:290] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.883064  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:34.888126  9772 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 0.
I20260709 13:16:34.888190  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
W20260709 13:16:34.889739  9736 leader_election.cc:343] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:34.890868  9662 leader_election.cc:343] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.892014  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:34.894711  9738 leader_election.cc:304] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:34.892958  9623 raft_consensus.cc:2468] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:34.897158  9664 leader_election.cc:304] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507
I20260709 13:16:34.898449  9931 raft_consensus.cc:2804] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.898954  9931 raft_consensus.cc:493] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.898670  9930 raft_consensus.cc:2804] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:34.899438  9931 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.899667  9930 raft_consensus.cc:493] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.900235  9930 raft_consensus.cc:3060] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.900321  9698 raft_consensus.cc:2468] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:34.893487  9942 leader_election.cc:290] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.899214  9915 tablet_bootstrap.cc:654] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:34.902909  9738 leader_election.cc:343] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.905450  9738 leader_election.cc:304] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac386aefbbe344588dea3a68164db507; no voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:34.910429  9587 leader_election.cc:304] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507
I20260709 13:16:34.907943  9942 raft_consensus.cc:2749] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:34.904261  9937 raft_consensus.cc:515] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.913229  9838 raft_consensus.cc:2804] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.921918  9937 leader_election.cc:290] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.924430  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:34.922737  9838 raft_consensus.cc:697] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:34.927495  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.928264  9697 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.925333  9771 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.930615  9914 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:34.930174  9930 raft_consensus.cc:515] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.931146  9914 ts_tablet_manager.cc:1403] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.013s
I20260709 13:16:34.935139  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:34.935894  9623 raft_consensus.cc:3060] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:34.939165  9931 raft_consensus.cc:515] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.941648  9697 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:34.943742  9587 leader_election.cc:304] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:34.944535  9937 raft_consensus.cc:2804] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.945077  9937 raft_consensus.cc:697] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:34.945459  9931 leader_election.cc:290] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:34.944557  9914 raft_consensus.cc:359] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.946717  9914 raft_consensus.cc:385] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.947082  9914 raft_consensus.cc:740] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.947896  9914 consensus_queue.cc:260] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.949582  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:34.950062  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:34.950640  9622 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:34.951020  9736 leader_election.cc:343] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.952360  9930 leader_election.cc:290] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:34.953039  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
W20260709 13:16:34.954653  9662 leader_election.cc:343] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:34.955343  9838 consensus_queue.cc:237] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.956126  9913 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:34.946307  9937 consensus_queue.cc:237] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.956691  9913 ts_tablet_manager.cc:1403] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.138s	user 0.019s	sys 0.021s
I20260709 13:16:34.961365  9771 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 1.
I20260709 13:16:34.961740  9914 ts_tablet_manager.cc:1434] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 13:16:34.965153  9915 tablet_bootstrap.cc:492] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:34.965775  9915 ts_tablet_manager.cc:1403] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.125s	user 0.012s	sys 0.005s
I20260709 13:16:34.969607  9622 raft_consensus.cc:2468] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:34.971045  9914 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: Bootstrap starting.
I20260709 13:16:34.969339  9913 raft_consensus.cc:359] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.960192  9623 raft_consensus.cc:2468] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:34.971805  9913 raft_consensus.cc:385] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.972148  9913 raft_consensus.cc:740] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.970376  9915 raft_consensus.cc:359] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.973409  9915 raft_consensus.cc:385] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:34.973919  9915 raft_consensus.cc:740] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.972245  9738 leader_election.cc:304] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:34.976507  9664 leader_election.cc:304] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507
I20260709 13:16:34.977627  9931 raft_consensus.cc:2804] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.977641  9930 raft_consensus.cc:2804] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:34.978529  9931 raft_consensus.cc:697] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:34.978552  9930 raft_consensus.cc:697] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:34.979673  9931 consensus_queue.cc:237] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.980003  9930 consensus_queue.cc:237] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.983700  9915 consensus_queue.cc:260] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:34.986452  9915 ts_tablet_manager.cc:1434] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:34.987684  9915 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Bootstrap starting.
I20260709 13:16:34.994323  9914 tablet_bootstrap.cc:654] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:34.995267  9913 consensus_queue.cc:260] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.000236  9915 tablet_bootstrap.cc:654] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:35.011771  9913 ts_tablet_manager.cc:1434] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.000s
I20260709 13:16:35.013654  9913 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:35.033556  9914 tablet_bootstrap.cc:492] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: No bootstrap required, opened a new log
I20260709 13:16:35.034216  9914 ts_tablet_manager.cc:1403] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.000s
I20260709 13:16:35.037487  9914 raft_consensus.cc:359] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.038254  9914 raft_consensus.cc:385] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:35.038566  9914 raft_consensus.cc:740] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.039630  9942 raft_consensus.cc:493] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.040271  9942 raft_consensus.cc:515] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.041292  9521 catalog_manager.cc:5697] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.043541  9521 catalog_manager.cc:5697] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.044978  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:35.045679  9697 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:35.045939  9519 catalog_manager.cc:5697] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.047194  9736 leader_election.cc:304] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.048012  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:35.048278  9901 raft_consensus.cc:2804] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:35.048712  9901 raft_consensus.cc:493] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.049124  9901 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.048714  9623 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 0.
I20260709 13:16:35.053733  9915 tablet_bootstrap.cc:492] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: No bootstrap required, opened a new log
I20260709 13:16:35.049367  9913 tablet_bootstrap.cc:654] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:35.054216  9915 ts_tablet_manager.cc:1403] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.010s
I20260709 13:16:35.055830  9942 leader_election.cc:290] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:35.063856  9901 raft_consensus.cc:515] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.063941  9914 consensus_queue.cc:260] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.066653  9901 leader_election.cc:290] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:35.068813  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.069367  9914 ts_tablet_manager.cc:1434] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.001s
I20260709 13:16:35.069515  9623 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.072048  9838 raft_consensus.cc:493] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.072528  9838 raft_consensus.cc:515] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.074400  9838 leader_election.cc:290] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.074966  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:35.075834  9697 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.075933  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:35.076701  9698 raft_consensus.cc:2393] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdf041618a254a1a9d5a2f014162cf90 in current term 1: Already voted for candidate ac386aefbbe344588dea3a68164db507 in this term.
I20260709 13:16:35.077873  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:35.079293  9587 leader_election.cc:304] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90; no voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.080758  9937 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.080904  9623 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:35.082121  9738 leader_election.cc:304] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:35.089989  9915 raft_consensus.cc:359] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.091079  9915 raft_consensus.cc:385] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:35.092589  9931 raft_consensus.cc:2804] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:35.093209  9518 catalog_manager.cc:5697] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:35.094466  9937 raft_consensus.cc:2749] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:35.095741  9931 raft_consensus.cc:697] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.097544  9697 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 1.
I20260709 13:16:35.096808  9931 consensus_queue.cc:237] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.095695  9518 catalog_manager.cc:5697] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:35.100483  9942 consensus_queue.cc:1048] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.091464  9915 raft_consensus.cc:740] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.106478  9915 consensus_queue.cc:260] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.107996  9930 raft_consensus.cc:493] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.108569  9930 raft_consensus.cc:515] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.109467  9913 tablet_bootstrap.cc:492] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:35.109983  9913 ts_tablet_manager.cc:1403] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.097s	user 0.014s	sys 0.032s
I20260709 13:16:35.110884  9930 leader_election.cc:290] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.112823  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:35.113006  9913 raft_consensus.cc:359] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.113695  9913 raft_consensus.cc:385] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:35.113976  9913 raft_consensus.cc:740] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.108970  9915 ts_tablet_manager.cc:1434] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.003s
I20260709 13:16:35.114738  9913 consensus_queue.cc:260] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.117487  9913 ts_tablet_manager.cc:1434] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:35.118347  9942 raft_consensus.cc:493] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.118669  9913 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: Bootstrap starting.
I20260709 13:16:35.119107  9942 raft_consensus.cc:515] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.122944  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:35.124133  9937 consensus_queue.cc:1048] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.125805  9913 tablet_bootstrap.cc:654] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:35.125797  9942 leader_election.cc:290] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:35.137127  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:35.138185  9623 raft_consensus.cc:2393] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1e5cfbaa43444d1b773156f512e2d11 in current term 1: Already voted for candidate ac386aefbbe344588dea3a68164db507 in this term.
I20260709 13:16:35.145287  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:35.146399  9697 raft_consensus.cc:2393] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac386aefbbe344588dea3a68164db507 in current term 1: Already voted for candidate bdf041618a254a1a9d5a2f014162cf90 in this term.
I20260709 13:16:35.147912  9736 leader_election.cc:304] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac386aefbbe344588dea3a68164db507; no voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.149091  9942 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.160372  9942 raft_consensus.cc:2749] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:35.162662  9937 consensus_queue.cc:1048] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.170373  9518 catalog_manager.cc:5697] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:35.160075  9664 leader_election.cc:304] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e5cfbaa43444d1b773156f512e2d11; no voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:35.192224  9930 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.198724  9930 raft_consensus.cc:2749] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:35.201812  9913 tablet_bootstrap.cc:492] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: No bootstrap required, opened a new log
I20260709 13:16:35.202301  9913 ts_tablet_manager.cc:1403] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: Time spent bootstrapping tablet: real 0.084s	user 0.036s	sys 0.038s
I20260709 13:16:35.205765  9913 raft_consensus.cc:359] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.209126  9913 raft_consensus.cc:385] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:35.209487  9913 raft_consensus.cc:740] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.210151  9913 consensus_queue.cc:260] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.208148  9942 consensus_queue.cc:1048] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.212194  9913 ts_tablet_manager.cc:1434] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:16:35.238857  9942 consensus_queue.cc:1048] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.251343  9931 consensus_queue.cc:1048] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.284956  9930 raft_consensus.cc:493] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.285454  9930 raft_consensus.cc:515] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.287231  9930 leader_election.cc:290] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.288123  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:35.288247  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507" is_pre_election: true
I20260709 13:16:35.288877  9623 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:35.288923  9771 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 0.
I20260709 13:16:35.290223  9664 leader_election.cc:304] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.290999  9900 raft_consensus.cc:2804] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:35.291273  9900 raft_consensus.cc:493] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.291584  9900 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.298013  9900 raft_consensus.cc:515] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.299759  9900 leader_election.cc:290] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.300534  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.300755  9771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.301088  9623 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.301350  9771 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.306480  9623 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:35.306950  9771 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 1.
I20260709 13:16:35.307587  9664 leader_election.cc:304] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.308405  9900 raft_consensus.cc:2804] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:35.308859  9900 raft_consensus.cc:697] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:35.309561  9900 consensus_queue.cc:237] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.321064  9521 catalog_manager.cc:5697] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 1 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.342646  9950 consensus_queue.cc:1048] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.347669  9950 consensus_queue.cc:1048] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.354290  9950 consensus_queue.cc:1048] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.363901  9950 consensus_queue.cc:1048] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:16:35.384872  9772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:35.385679  9772 raft_consensus.cc:606] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:35.388926  9931 raft_consensus.cc:493] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.389393  9931 raft_consensus.cc:515] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.391646  9931 leader_election.cc:290] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:35.392313  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf041618a254a1a9d5a2f014162cf90" is_pre_election: true
I20260709 13:16:35.392782  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0371001f236f47a397a0e216a7abd61a" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11" is_pre_election: true
I20260709 13:16:35.393160  9623 raft_consensus.cc:2393] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac386aefbbe344588dea3a68164db507 in current term 1: Already voted for candidate c1e5cfbaa43444d1b773156f512e2d11 in this term.
I20260709 13:16:35.392405  9772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:35.393712  9772 raft_consensus.cc:606] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS bdf041618a254a1a9d5a2f014162cf90
I20260709 13:16:35.394435  9736 leader_election.cc:304] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac386aefbbe344588dea3a68164db507; no voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.395201  9931 raft_consensus.cc:3060] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:35.399961  9623 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:45296
I20260709 13:16:35.400674  9623 raft_consensus.cc:606] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.401854  9931 raft_consensus.cc:2749] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:35.406208  9772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:35.406903  9772 raft_consensus.cc:606] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.414891  9772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
mode: GRACEFUL
new_leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:50294
I20260709 13:16:35.415531  9772 raft_consensus.cc:606] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Received request to transfer leadership to TS c1e5cfbaa43444d1b773156f512e2d11
I20260709 13:16:35.420444  9697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:49028
I20260709 13:16:35.421042  9697 raft_consensus.cc:606] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:35.453369  9900 consensus_queue.cc:1048] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.465375  9900 consensus_queue.cc:1048] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:35.473793  9901 consensus_queue.cc:1048] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.492908  9931 consensus_queue.cc:1048] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.504012  9931 raft_consensus.cc:993] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: : Instructing follower bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:35.506637  9942 raft_consensus.cc:1081] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:35.508565  9623 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:45290
I20260709 13:16:35.509123  9623 raft_consensus.cc:493] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.509516  9623 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.514369  9623 raft_consensus.cc:515] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.516191  9623 leader_election.cc:290] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.517104  9697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:35.517249  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d016eb1f6f184f21be6a986e4086b5ec" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.517963  9772 raft_consensus.cc:3055] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.518323  9772 raft_consensus.cc:740] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.518466  9697 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.519102  9772 consensus_queue.cc:260] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.520109  9772 raft_consensus.cc:3060] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.525256  9697 raft_consensus.cc:2468] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:35.525487  9931 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.526456  9587 leader_election.cc:304] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.527175  9950 raft_consensus.cc:2804] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.527734  9950 raft_consensus.cc:697] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:35.527793  9772 raft_consensus.cc:2468] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:35.528761  9950 consensus_queue.cc:237] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.536932  9942 raft_consensus.cc:993] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.537503  9942 raft_consensus.cc:1081] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.540583  9698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49024
I20260709 13:16:35.541180  9698 raft_consensus.cc:493] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.541639  9698 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.540827  9521 catalog_manager.cc:5697] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.547711  9931 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.557442  9698 raft_consensus.cc:515] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.560909  9697 raft_consensus.cc:1240] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Rejecting Update request from peer ac386aefbbe344588dea3a68164db507 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:35.564215  9698 leader_election.cc:290] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.564322  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.565002  9623 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.565290  9931 consensus_queue.cc:1059] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.565781  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.566356  9901 raft_consensus.cc:3055] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.566743  9901 raft_consensus.cc:740] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.567582  9901 consensus_queue.cc:260] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.569024  9901 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.571632  9623 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.572739  9664 leader_election.cc:304] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.573483  9900 raft_consensus.cc:2804] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.574024  9900 raft_consensus.cc:697] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:35.575002  9900 consensus_queue.cc:237] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.583753  9772 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.583343  9521 catalog_manager.cc:5697] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.655169  9931 raft_consensus.cc:993] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.655671  9901 raft_consensus.cc:1081] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.657088  9698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49024
I20260709 13:16:35.657585  9698 raft_consensus.cc:493] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.657930  9698 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.663321  9698 raft_consensus.cc:515] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.664794  9698 leader_election.cc:290] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.665591  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.666049  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf55527ffda46a3b33fe2c23a24d553" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.666286  9623 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.666627  9772 raft_consensus.cc:3055] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.666942  9772 raft_consensus.cc:740] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.667733  9772 consensus_queue.cc:260] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.668915  9772 raft_consensus.cc:3060] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.672809  9623 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.674121  9664 leader_election.cc:304] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.674891  9900 raft_consensus.cc:2804] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.675359  9772 raft_consensus.cc:2468] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.675432  9900 raft_consensus.cc:697] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:35.676803  9900 consensus_queue.cc:237] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.678267  9942 raft_consensus.cc:993] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: : Instructing follower bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:35.678772  9901 raft_consensus.cc:1081] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Signalling peer bdf041618a254a1a9d5a2f014162cf90 to start an election
I20260709 13:16:35.680388  9623 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
 from {username='slave'} at 127.0.0.1:45290
I20260709 13:16:35.680898  9623 raft_consensus.cc:493] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.681212  9623 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.684230  9521 catalog_manager.cc:5697] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.685858  9623 raft_consensus.cc:515] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.687791  9623 leader_election.cc:290] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Requested vote from peers c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.688586  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:35.689203  9698 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.689182  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f60df8c06c94fd5a561bc5497790e72" candidate_uuid: "bdf041618a254a1a9d5a2f014162cf90" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.690001  9772 raft_consensus.cc:3055] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.691248  9772 raft_consensus.cc:740] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.691965  9772 consensus_queue.cc:260] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.692932  9772 raft_consensus.cc:3060] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.695822  9698 raft_consensus.cc:2468] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:35.697759  9587 leader_election.cc:304] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf041618a254a1a9d5a2f014162cf90, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.698527  9950 raft_consensus.cc:2804] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.698612  9772 raft_consensus.cc:2468] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf041618a254a1a9d5a2f014162cf90 in term 2.
I20260709 13:16:35.698967  9950 raft_consensus.cc:697] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Leader. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:35.699846  9950 consensus_queue.cc:237] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.707165  9521 catalog_manager.cc:5697] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 reported cstate change: term changed from 1 to 2, leader changed from ac386aefbbe344588dea3a68164db507 (127.31.120.3) to bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "bdf041618a254a1a9d5a2f014162cf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.764719  9950 raft_consensus.cc:993] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.765219  9937 raft_consensus.cc:1081] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Signalling peer c1e5cfbaa43444d1b773156f512e2d11 to start an election
I20260709 13:16:35.766621  9698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
 from {username='slave'} at 127.0.0.1:49010
I20260709 13:16:35.767067  9698 raft_consensus.cc:493] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.767344  9698 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.771839  9698 raft_consensus.cc:515] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.773258  9698 leader_election.cc:290] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), ac386aefbbe344588dea3a68164db507 (127.31.120.3:42851)
I20260709 13:16:35.774156  9623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.774421  9772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d1254e35c848718bd9ff4c211bdd9c" candidate_uuid: "c1e5cfbaa43444d1b773156f512e2d11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac386aefbbe344588dea3a68164db507"
I20260709 13:16:35.774708  9623 raft_consensus.cc:3055] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.775007  9772 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.775049  9623 raft_consensus.cc:740] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:35.775908  9623 consensus_queue.cc:260] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.776783  9623 raft_consensus.cc:3060] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.779978  9772 raft_consensus.cc:2468] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.781204  9623 raft_consensus.cc:2468] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e5cfbaa43444d1b773156f512e2d11 in term 2.
I20260709 13:16:35.781236  9662 leader_election.cc:304] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:35.782089  9900 raft_consensus.cc:2804] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.782511  9900 raft_consensus.cc:697] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Leader. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:35.783288  9900 consensus_queue.cc:237] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.790778  9521 catalog_manager.cc:5697] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 reported cstate change: term changed from 1 to 2, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2). New cstate: current_term: 2 leader_uuid: "c1e5cfbaa43444d1b773156f512e2d11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.809600  9900 consensus_queue.cc:1048] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.816825  9930 raft_consensus.cc:993] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:35.817361  9977 raft_consensus.cc:1081] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:35.819178  9771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50286
I20260709 13:16:35.820396  9771 raft_consensus.cc:493] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.820824  9771 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.823072  9977 consensus_queue.cc:1048] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:35.828713  9771 raft_consensus.cc:515] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.831642  9622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:35.832283  9622 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.838884  9622 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:35.839928  9738 leader_election.cc:304] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:35.840157  9771 leader_election.cc:290] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:35.840747  9901 raft_consensus.cc:2804] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:35.841245  9901 raft_consensus.cc:697] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:35.842000  9901 consensus_queue.cc:237] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.845436  9698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e27fb303e045ae8e7cc648bacfdd77" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:35.846065  9698 raft_consensus.cc:3055] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:35.846473  9698 raft_consensus.cc:740] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:35.847188  9698 consensus_queue.cc:260] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:35.848510  9698 raft_consensus.cc:3060] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.849269  9519 catalog_manager.cc:5697] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 1 to 2, leader changed from c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:35.853466  9698 raft_consensus.cc:2468] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:35.979964  9698 raft_consensus.cc:1275] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:35.981235  9937 consensus_queue.cc:1048] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.990419  9771 raft_consensus.cc:1275] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:35.992820  9985 consensus_queue.cc:1048] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.081130  9698 raft_consensus.cc:1275] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.082296  9985 consensus_queue.cc:1048] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.091274  9771 raft_consensus.cc:1275] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer bdf041618a254a1a9d5a2f014162cf90: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.092742  9985 consensus_queue.cc:1048] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.114612  9622 raft_consensus.cc:1275] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.116066  9977 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.124945  9771 raft_consensus.cc:1275] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.126183  9930 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.215916  9771 raft_consensus.cc:1275] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.217121  9930 consensus_queue.cc:1048] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.220238  9622 raft_consensus.cc:1275] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.221895  9930 consensus_queue.cc:1048] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:36.300458  9622 raft_consensus.cc:1275] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.301661  9942 consensus_queue.cc:1048] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.309281  9698 raft_consensus.cc:1275] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.312667  9988 consensus_queue.cc:1048] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.334981  9771 raft_consensus.cc:1275] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.336196  9977 consensus_queue.cc:1048] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.344946  9622 raft_consensus.cc:1275] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer c1e5cfbaa43444d1b773156f512e2d11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:36.346319  9900 consensus_queue.cc:1048] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:38.425134  9519 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58680:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:16:38.447482  9519 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58680:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:16:38.483160 32224 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:16:38.566325 10006 log.cc:927] Time spent T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:16:38.745671 10044 tablet.cc:1724] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.677467 10033 tablet.cc:1724] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.750034 10046 tablet.cc:1724] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.760664 10047 tablet.cc:1724] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:16:38.777063  9649 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:7ed743650b834413a19d90748f427f58)
W20260709 13:16:38.793680  9649 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:7ed743650b834413a19d90748f427f58)
W20260709 13:16:38.794713  9649 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:7ed743650b834413a19d90748f427f58)
I20260709 13:16:38.813606 10054 tablet.cc:1724] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.817942 10053 tablet.cc:1724] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.820591 10056 tablet.cc:1724] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:16:38.847704 10036 log.cc:927] Time spent T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 13:16:38.850764 10057 tablet.cc:1724] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.855154 10059 tablet.cc:1724] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:16:38.853885  9587 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:35081, user_credentials={real_user=slave}} blocked reactor thread for 106396us
I20260709 13:16:38.836568 10055 tablet.cc:1724] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.884357 10058 tablet.cc:1724] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.893738  9695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:57590
I20260709 13:16:38.894644  9695 raft_consensus.cc:606] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:38.909258 10046 tablet.cc:1724] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.830688 10054 tablet.cc:1724] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.920542 10047 tablet.cc:1724] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.933768 10061 tablet.cc:1724] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.936651 10044 tablet.cc:1724] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.911940 10033 tablet.cc:1724] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.951272 10055 tablet.cc:1724] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.959863 10057 tablet.cc:1724] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.965857 10056 tablet.cc:1724] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.974896 10059 tablet.cc:1724] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.977289 10044 tablet.cc:1724] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.995941 10053 tablet.cc:1724] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.997179 10047 tablet.cc:1724] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:39.001523 10057 tablet.cc:1724] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:38.975284 10033 tablet.cc:1724] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:39.037153  9621 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ed743650b834413a19d90748f427f58"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:38450
I20260709 13:16:39.036859 10057 tablet.cc:1724] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:39.037869  9621 raft_consensus.cc:606] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:39.041086 32224 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:16:39.041675 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:39.056857  9696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf"
dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:57590
I20260709 13:16:39.057546  9696 raft_consensus.cc:606] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:39.060786  9621 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e"
dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
mode: GRACEFUL
new_leader_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:38450
I20260709 13:16:39.061414  9621 raft_consensus.cc:606] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Received request to transfer leadership to TS ac386aefbbe344588dea3a68164db507
I20260709 13:16:39.060420  9517 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: soft_deleted_table [id=840a67c9ba24414d9a754136c4472599] alter complete (version 1)
I20260709 13:16:39.062615 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:39.063179 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:39.167044  9985 raft_consensus.cc:993] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.167577 10060 raft_consensus.cc:1081] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.169056  9768 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ed743650b834413a19d90748f427f58"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50270
I20260709 13:16:39.169500  9768 raft_consensus.cc:493] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:39.169759  9768 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:39.175016  9768 raft_consensus.cc:515] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.176693  9768 leader_election.cc:290] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:39.177728  9621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:39.177901  9696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed743650b834413a19d90748f427f58" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:39.178606  9621 raft_consensus.cc:3055] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:39.178682  9696 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:39.178928  9621 raft_consensus.cc:740] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:39.179934  9621 consensus_queue.cc:260] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.181006  9621 raft_consensus.cc:3060] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:39.184509  9696 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:39.185714  9736 leader_election.cc:304] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, c1e5cfbaa43444d1b773156f512e2d11; no voters: 
I20260709 13:16:39.186587 10026 raft_consensus.cc:2804] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:39.187103 10026 raft_consensus.cc:697] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:39.187055  9621 raft_consensus.cc:2468] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 2.
I20260709 13:16:39.188186 10026 consensus_queue.cc:237] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.196346  9520 catalog_manager.cc:5697] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 1 to 2, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.314289 10029 raft_consensus.cc:993] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.314911 10024 raft_consensus.cc:1081] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.316550  9768 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50286
I20260709 13:16:39.317113  9768 raft_consensus.cc:493] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:39.317493  9768 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.322568  9768 raft_consensus.cc:515] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.324286  9768 leader_election.cc:290] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:39.325240  9621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:39.325387  9695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54d3de9f4fe949d79e675ef0630c1398" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:39.325923  9621 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.326004  9695 raft_consensus.cc:3055] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.326414  9695 raft_consensus.cc:740] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:39.327061  9695 consensus_queue.cc:260] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.328049  9695 raft_consensus.cc:3060] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.331112  9621 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.332336  9738 leader_election.cc:304] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:39.333076 10026 raft_consensus.cc:2804] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:39.333482  9695 raft_consensus.cc:2468] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.333599 10026 raft_consensus.cc:697] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:39.334609 10026 consensus_queue.cc:237] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.341648  9520 catalog_manager.cc:5697] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 2 to 3, leader changed from c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.363085 10029 raft_consensus.cc:993] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.363545 10024 raft_consensus.cc:1081] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.365051  9768 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50286
I20260709 13:16:39.365489  9768 raft_consensus.cc:493] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:39.365733  9768 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.371243  9768 raft_consensus.cc:515] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.373273  9768 leader_election.cc:290] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:39.374208  9695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:39.374389  9621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5455f6674b914b5f87bdd3652e6a65bf" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:39.374863  9695 raft_consensus.cc:3055] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.375057  9621 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.375191  9695 raft_consensus.cc:740] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1e5cfbaa43444d1b773156f512e2d11, State: Running, Role: LEADER
I20260709 13:16:39.376080  9695 consensus_queue.cc:260] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.377080  9695 raft_consensus.cc:3060] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.380514  9621 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.381999  9738 leader_election.cc:304] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:39.382794 10026 raft_consensus.cc:2804] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:39.383266 10026 raft_consensus.cc:697] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:39.383405  9695 raft_consensus.cc:2468] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.384223 10026 consensus_queue.cc:237] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } }
I20260709 13:16:39.385825 10060 raft_consensus.cc:993] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: : Instructing follower ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.386368  9985 raft_consensus.cc:1081] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Signalling peer ac386aefbbe344588dea3a68164db507 to start an election
I20260709 13:16:39.388072  9769 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e"
dest_uuid: "ac386aefbbe344588dea3a68164db507"
 from {username='slave'} at 127.0.0.1:50270
I20260709 13:16:39.388550  9769 raft_consensus.cc:493] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:39.388861  9769 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.394830  9520 catalog_manager.cc:5697] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 2 to 3, leader changed from c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:39.396333  9769 raft_consensus.cc:515] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:39.398424  9769 leader_election.cc:290] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Requested vote from peers bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853), c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081)
I20260709 13:16:39.399681  9620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bdf041618a254a1a9d5a2f014162cf90"
I20260709 13:16:39.400291  9620 raft_consensus.cc:3055] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:39.400602  9620 raft_consensus.cc:740] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bdf041618a254a1a9d5a2f014162cf90, State: Running, Role: LEADER
I20260709 13:16:39.401409  9620 consensus_queue.cc:260] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:39.402441  9620 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.404680  9695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc40916a38ed4de5adbc19eec6314c4e" candidate_uuid: "ac386aefbbe344588dea3a68164db507" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c1e5cfbaa43444d1b773156f512e2d11"
I20260709 13:16:39.405457  9695 raft_consensus.cc:3060] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:39.408308  9620 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.409312  9738 leader_election.cc:304] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac386aefbbe344588dea3a68164db507, bdf041618a254a1a9d5a2f014162cf90; no voters: 
I20260709 13:16:39.410030 10026 raft_consensus.cc:2804] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:39.410565 10026 raft_consensus.cc:697] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Becoming Leader. State: Replica: ac386aefbbe344588dea3a68164db507, State: Running, Role: LEADER
I20260709 13:16:39.410686  9695 raft_consensus.cc:2468] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac386aefbbe344588dea3a68164db507 in term 3.
I20260709 13:16:39.411499 10026 consensus_queue.cc:237] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } }
I20260709 13:16:39.421625  9520 catalog_manager.cc:5697] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 reported cstate change: term changed from 2 to 3, leader changed from bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1) to ac386aefbbe344588dea3a68164db507 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "ac386aefbbe344588dea3a68164db507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac386aefbbe344588dea3a68164db507" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:39.736266  9695 raft_consensus.cc:1275] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:39.737672 10026 consensus_queue.cc:1048] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:39.747092  9620 raft_consensus.cc:1275] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:39.748849  9988 consensus_queue.cc:1048] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:39.780433  9620 raft_consensus.cc:1275] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:39.781659 10026 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:39.789184  9695 raft_consensus.cc:1275] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:39.790591 10071 consensus_queue.cc:1048] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:39.848263  9695 raft_consensus.cc:1275] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:39.850194 10071 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:39.864035  9620 raft_consensus.cc:1275] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:39.865631  9988 consensus_queue.cc:1048] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:39.877774  9695 raft_consensus.cc:1275] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:39.879093 10073 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e5cfbaa43444d1b773156f512e2d11" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:39.888753  9620 raft_consensus.cc:1275] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Refusing update from remote peer ac386aefbbe344588dea3a68164db507: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:39.890384 10073 consensus_queue.cc:1048] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf041618a254a1a9d5a2f014162cf90" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:40.123719 32224 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:16:40.129287 32224 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:16:40.129678 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:40.134933 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:16:40.135326 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:41.164072 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
W20260709 13:16:41.187942  9738 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 13:16:41.191768 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:16:41.192003  9738 consensus_peers.cc:597] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.192621 32224 tablet_replica.cc:333] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.193817 32224 raft_consensus.cc:2243] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.194329 32224 raft_consensus.cc:2272] T 0371001f236f47a397a0e216a7abd61a P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.196785 32224 tablet_replica.cc:333] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.197806 32224 raft_consensus.cc:2243] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.198724 32224 raft_consensus.cc:2272] T d016eb1f6f184f21be6a986e4086b5ec P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.201051 32224 tablet_replica.cc:333] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.201632 32224 raft_consensus.cc:2243] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.202378 32224 raft_consensus.cc:2272] T 881ffabead614781aee7016272cd256f P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.205054 32224 tablet_replica.cc:333] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.205832 32224 raft_consensus.cc:2243] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.206519 32224 raft_consensus.cc:2272] T d1787d4051c14afcac84c33f9d4ff691 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.209461 32224 tablet_replica.cc:333] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.210219 32224 raft_consensus.cc:2243] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.211144 32224 raft_consensus.cc:2272] T 6f60df8c06c94fd5a561bc5497790e72 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.213469 32224 tablet_replica.cc:333] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.214138 32224 raft_consensus.cc:2243] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.214609 32224 raft_consensus.cc:2272] T cc40916a38ed4de5adbc19eec6314c4e P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.217608 32224 tablet_replica.cc:333] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.218204 32224 raft_consensus.cc:2243] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.219343 32224 raft_consensus.cc:2272] T e40a69de06904e52a6642505b5a42db1 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.221691 32224 tablet_replica.cc:333] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.222358 32224 raft_consensus.cc:2243] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.223192 32224 raft_consensus.cc:2272] T 96316755af8c4c9dac15c7556b76e4cc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.225410 32224 tablet_replica.cc:333] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.226002 32224 raft_consensus.cc:2243] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.226504 32224 raft_consensus.cc:2272] T 5a4299f51a58415a93d06cf2685ea63a P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.228754 32224 tablet_replica.cc:333] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.229372 32224 raft_consensus.cc:2243] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.229844 32224 raft_consensus.cc:2272] T 7ed743650b834413a19d90748f427f58 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.231945 32224 tablet_replica.cc:333] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.232586 32224 raft_consensus.cc:2243] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.233074 32224 raft_consensus.cc:2272] T 43319ce311c54054990dde238848975d P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.235205 32224 tablet_replica.cc:333] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.235831 32224 raft_consensus.cc:2243] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.236425 32224 raft_consensus.cc:2272] T 54d3de9f4fe949d79e675ef0630c1398 P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.238356 32224 tablet_replica.cc:333] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.239005 32224 raft_consensus.cc:2243] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.239504 32224 raft_consensus.cc:2272] T 39d1254e35c848718bd9ff4c211bdd9c P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.242285 32224 tablet_replica.cc:333] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.242883 32224 raft_consensus.cc:2243] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.243449 32224 raft_consensus.cc:2272] T 16e27fb303e045ae8e7cc648bacfdd77 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.245388 32224 tablet_replica.cc:333] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.246062 32224 raft_consensus.cc:2243] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.246694 32224 raft_consensus.cc:2272] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.249722 32224 tablet_replica.cc:333] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.250512 32224 raft_consensus.cc:2243] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.251107 32224 raft_consensus.cc:2272] T ecf55527ffda46a3b33fe2c23a24d553 P bdf041618a254a1a9d5a2f014162cf90 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.253999 32224 tablet_replica.cc:333] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.254624 32224 raft_consensus.cc:2243] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.255533 32224 raft_consensus.cc:2272] T 2dc8fd062d764891b4df49bafb99de16 P bdf041618a254a1a9d5a2f014162cf90 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:41.257280  9664 consensus_peers.cc:597] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.258551 32224 tablet_replica.cc:333] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90: stopping tablet replica
I20260709 13:16:41.259195 32224 raft_consensus.cc:2243] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:41.259678  9738 consensus_peers.cc:597] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.259758 32224 raft_consensus.cc:2272] T 5455f6674b914b5f87bdd3652e6a65bf P bdf041618a254a1a9d5a2f014162cf90 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.303054 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
W20260709 13:16:41.318770  9664 consensus_peers.cc:597] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:41.329341  9664 consensus_peers.cc:597] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.341931 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:16:41.351145  9736 consensus_peers.cc:597] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 -> Peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Couldn't send request to peer c1e5cfbaa43444d1b773156f512e2d11. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.380239 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:41.381058 32224 tablet_replica.cc:333] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.381666 32224 raft_consensus.cc:2243] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.382144 32224 raft_consensus.cc:2272] T d016eb1f6f184f21be6a986e4086b5ec P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.384184 32224 tablet_replica.cc:333] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.384781 32224 raft_consensus.cc:2243] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.385586 32224 raft_consensus.cc:2272] T 39d1254e35c848718bd9ff4c211bdd9c P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.387683 32224 tablet_replica.cc:333] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.388283 32224 raft_consensus.cc:2243] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.388708 32224 raft_consensus.cc:2272] T 54d3de9f4fe949d79e675ef0630c1398 P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.390771 32224 tablet_replica.cc:333] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.391331 32224 raft_consensus.cc:2243] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.391819 32224 raft_consensus.cc:2272] T d1787d4051c14afcac84c33f9d4ff691 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.393991 32224 tablet_replica.cc:333] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.394610 32224 raft_consensus.cc:2243] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.395056 32224 raft_consensus.cc:2272] T 5455f6674b914b5f87bdd3652e6a65bf P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.397239 32224 tablet_replica.cc:333] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.397801 32224 raft_consensus.cc:2243] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.398289 32224 raft_consensus.cc:2272] T 96316755af8c4c9dac15c7556b76e4cc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.400573 32224 tablet_replica.cc:333] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.401145 32224 raft_consensus.cc:2243] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.401592 32224 raft_consensus.cc:2272] T e40a69de06904e52a6642505b5a42db1 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.404799 32224 tablet_replica.cc:333] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.405520 32224 raft_consensus.cc:2243] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.406103 32224 raft_consensus.cc:2272] T 16e27fb303e045ae8e7cc648bacfdd77 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.408293 32224 tablet_replica.cc:333] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.408982 32224 raft_consensus.cc:2243] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.409916 32224 raft_consensus.cc:2272] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.412637 32224 tablet_replica.cc:333] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.413277 32224 raft_consensus.cc:2243] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.413690 32224 raft_consensus.cc:2272] T 6f60df8c06c94fd5a561bc5497790e72 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.415571 32224 tablet_replica.cc:333] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.416105 32224 raft_consensus.cc:2243] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.416476 32224 raft_consensus.cc:2272] T cc40916a38ed4de5adbc19eec6314c4e P c1e5cfbaa43444d1b773156f512e2d11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.418771 32224 tablet_replica.cc:333] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.419282 32224 raft_consensus.cc:2243] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.419778 32224 raft_consensus.cc:2272] T 7ed743650b834413a19d90748f427f58 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:41.421119  9736 consensus_peers.cc:597] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 -> Peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Couldn't send request to peer c1e5cfbaa43444d1b773156f512e2d11. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.421844 32224 tablet_replica.cc:333] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.422447 32224 raft_consensus.cc:2243] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.423189 32224 raft_consensus.cc:2272] T 43319ce311c54054990dde238848975d P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.425302 32224 tablet_replica.cc:333] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.425822 32224 raft_consensus.cc:2243] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.426585 32224 raft_consensus.cc:2272] T ecf55527ffda46a3b33fe2c23a24d553 P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.428536 32224 tablet_replica.cc:333] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.429065 32224 raft_consensus.cc:2243] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.429502 32224 raft_consensus.cc:2272] T 2dc8fd062d764891b4df49bafb99de16 P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.431461 32224 tablet_replica.cc:333] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.431973 32224 raft_consensus.cc:2243] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.432708 32224 raft_consensus.cc:2272] T 0371001f236f47a397a0e216a7abd61a P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.434612 32224 tablet_replica.cc:333] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.435092 32224 raft_consensus.cc:2243] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.435916 32224 raft_consensus.cc:2272] T 5a4299f51a58415a93d06cf2685ea63a P c1e5cfbaa43444d1b773156f512e2d11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.437883 32224 tablet_replica.cc:333] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11: stopping tablet replica
I20260709 13:16:41.438326 32224 raft_consensus.cc:2243] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.438730 32224 raft_consensus.cc:2272] T 881ffabead614781aee7016272cd256f P c1e5cfbaa43444d1b773156f512e2d11 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:41.459704  9738 consensus_peers.cc:597] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.480038 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
W20260709 13:16:41.509392  9738 consensus_peers.cc:597] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:41.510716  9736 consensus_peers.cc:597] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 -> Peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Couldn't send request to peer c1e5cfbaa43444d1b773156f512e2d11. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35081: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.522004 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
W20260709 13:16:41.548518  9738 consensus_peers.cc:597] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 -> Peer bdf041618a254a1a9d5a2f014162cf90 (127.31.120.1:40853): Couldn't send request to peer bdf041618a254a1a9d5a2f014162cf90. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:40853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:41.549451  9736 consensus_peers.cc:597] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 -> Peer c1e5cfbaa43444d1b773156f512e2d11 (127.31.120.2:35081): Couldn't send request to peer c1e5cfbaa43444d1b773156f512e2d11. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:35081: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:41.555075 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:41.555821 32224 tablet_replica.cc:333] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.556429 32224 raft_consensus.cc:2243] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.556900 32224 raft_consensus.cc:2272] T 0371001f236f47a397a0e216a7abd61a P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.558713 32224 tablet_replica.cc:333] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.559265 32224 raft_consensus.cc:2243] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.559746 32224 raft_consensus.cc:2272] T 39d1254e35c848718bd9ff4c211bdd9c P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.561444 32224 tablet_replica.cc:333] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.561966 32224 raft_consensus.cc:2243] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.562726 32224 raft_consensus.cc:2272] T 54d3de9f4fe949d79e675ef0630c1398 P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.564553 32224 tablet_replica.cc:333] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.565092 32224 raft_consensus.cc:2243] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.565518 32224 raft_consensus.cc:2272] T d016eb1f6f184f21be6a986e4086b5ec P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.567294 32224 tablet_replica.cc:333] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.567883 32224 raft_consensus.cc:2243] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.568301 32224 raft_consensus.cc:2272] T 43319ce311c54054990dde238848975d P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.570381 32224 tablet_replica.cc:333] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.570904 32224 raft_consensus.cc:2243] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.571718 32224 raft_consensus.cc:2272] T 7ed743650b834413a19d90748f427f58 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.573551 32224 tablet_replica.cc:333] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.574090 32224 raft_consensus.cc:2243] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.574523 32224 raft_consensus.cc:2272] T ecf55527ffda46a3b33fe2c23a24d553 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.576390 32224 tablet_replica.cc:333] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.576910 32224 raft_consensus.cc:2243] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.577320 32224 raft_consensus.cc:2272] T 2dc8fd062d764891b4df49bafb99de16 P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.579208 32224 tablet_replica.cc:333] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.579807 32224 raft_consensus.cc:2243] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.580611 32224 raft_consensus.cc:2272] T d1787d4051c14afcac84c33f9d4ff691 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.582595 32224 tablet_replica.cc:333] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.583133 32224 raft_consensus.cc:2243] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.583562 32224 raft_consensus.cc:2272] T 5a4299f51a58415a93d06cf2685ea63a P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.585520 32224 tablet_replica.cc:333] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.585981 32224 raft_consensus.cc:2243] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.586359 32224 raft_consensus.cc:2272] T 6f60df8c06c94fd5a561bc5497790e72 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.588238 32224 tablet_replica.cc:333] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.588739 32224 raft_consensus.cc:2243] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.589385 32224 raft_consensus.cc:2272] T cc40916a38ed4de5adbc19eec6314c4e P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.591811 32224 tablet_replica.cc:333] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.592377 32224 raft_consensus.cc:2243] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.592869 32224 raft_consensus.cc:2272] T 96316755af8c4c9dac15c7556b76e4cc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.594883 32224 tablet_replica.cc:333] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.595376 32224 raft_consensus.cc:2243] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.595867 32224 raft_consensus.cc:2272] T e40a69de06904e52a6642505b5a42db1 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.597842 32224 tablet_replica.cc:333] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.598342 32224 raft_consensus.cc:2243] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.599095 32224 raft_consensus.cc:2272] T 16e27fb303e045ae8e7cc648bacfdd77 P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.601094 32224 tablet_replica.cc:333] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.601681 32224 raft_consensus.cc:2243] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.602128 32224 raft_consensus.cc:2272] T a3baf9fcdf364ef0b6d9fbe7e4b22dfc P ac386aefbbe344588dea3a68164db507 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.604167 32224 tablet_replica.cc:333] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.604773 32224 raft_consensus.cc:2243] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.605189 32224 raft_consensus.cc:2272] T 881ffabead614781aee7016272cd256f P ac386aefbbe344588dea3a68164db507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.607087 32224 tablet_replica.cc:333] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507: stopping tablet replica
I20260709 13:16:41.607547 32224 raft_consensus.cc:2243] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.608366 32224 raft_consensus.cc:2272] T 5455f6674b914b5f87bdd3652e6a65bf P ac386aefbbe344588dea3a68164db507 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.658147 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:16:41.689962 32224 master.cc:562] Master@127.31.120.62:32987 shutting down...
I20260709 13:16:41.710847 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:41.711503 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.712095 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3eb040a608864513ac4117b392d009d6: stopping tablet replica
I20260709 13:16:41.732679 32224 master.cc:584] Master@127.31.120.62:32987 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12504 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:16:41.790129 32224 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.31.120.62:42313
I20260709 13:16:41.791304 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:41.798219 10086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:41.798269 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:41.798632 10089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:41.800685 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:41.801604 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:41.801800 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:41.801915 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783603001801904 us; error 0 us; skew 500 ppm
I20260709 13:16:41.802350 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:41.804728 32224 webserver.cc:533] Webserver started at http://127.31.120.62:44057/ using document root <none> and password file <none>
I20260709 13:16:41.805194 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:41.805343 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:41.805548 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:41.806650 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/master-0-root/instance:
uuid: "b5eaf6252ad5425292938d766398107d"
format_stamp: "Formatted at 2026-07-09 13:16:41 on dist-test-slave-pm98"
I20260709 13:16:41.810956 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:16:41.814410 10094 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:41.815289 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:41.815536 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/master-0-root
uuid: "b5eaf6252ad5425292938d766398107d"
format_stamp: "Formatted at 2026-07-09 13:16:41 on dist-test-slave-pm98"
I20260709 13:16:41.815864 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:41.836994 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:41.838164 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:41.881515 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.62:42313
I20260709 13:16:41.881621 10145 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.62:42313 every 8 connection(s)
I20260709 13:16:41.885834 10146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:41.896757 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: Bootstrap starting.
I20260709 13:16:41.901674 10146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:41.906132 10146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: No bootstrap required, opened a new log
I20260709 13:16:41.908427 10146 raft_consensus.cc:359] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER }
I20260709 13:16:41.908824 10146 raft_consensus.cc:385] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:41.909056 10146 raft_consensus.cc:740] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5eaf6252ad5425292938d766398107d, State: Initialized, Role: FOLLOWER
I20260709 13:16:41.909679 10146 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER }
I20260709 13:16:41.910125 10146 raft_consensus.cc:399] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:41.910321 10146 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:41.910604 10146 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:41.915658 10146 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER }
I20260709 13:16:41.916174 10146 leader_election.cc:304] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5eaf6252ad5425292938d766398107d; no voters: 
I20260709 13:16:41.917289 10146 leader_election.cc:290] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:41.917639 10149 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:41.919466 10149 raft_consensus.cc:697] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 1 LEADER]: Becoming Leader. State: Replica: b5eaf6252ad5425292938d766398107d, State: Running, Role: LEADER
I20260709 13:16:41.920253 10146 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:41.920241 10149 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER }
I20260709 13:16:41.925343 10151 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5eaf6252ad5425292938d766398107d. Latest consensus state: current_term: 1 leader_uuid: "b5eaf6252ad5425292938d766398107d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER } }
I20260709 13:16:41.925530 10150 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5eaf6252ad5425292938d766398107d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5eaf6252ad5425292938d766398107d" member_type: VOTER } }
I20260709 13:16:41.926192 10151 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:41.926403 10150 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:41.931113 10157 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:41.936482 10157 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:41.937567 32224 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:41.945351 10157 catalog_manager.cc:1383] Generated new cluster ID: 003635b98de043ddb179f9ba8b08947b
I20260709 13:16:41.945642 10157 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:41.959898 10157 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:41.961192 10157 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:41.975430 10157 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: Generated new TSK 0
I20260709 13:16:41.976079 10157 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:42.004588 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:42.010939 10167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:42.012153 10168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:42.014546 10170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:42.014609 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:42.015827 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:42.016021 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:42.016214 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783603002016196 us; error 0 us; skew 500 ppm
I20260709 13:16:42.016718 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:42.019336 32224 webserver.cc:533] Webserver started at http://127.31.120.1:39611/ using document root <none> and password file <none>
I20260709 13:16:42.019881 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:42.020063 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:42.020438 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:42.021515 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-0-root/instance:
uuid: "57da9927cb614902a03b2f62dca01939"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.026190 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:16:42.030352 10175 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.031368 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:16:42.031706 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-0-root
uuid: "57da9927cb614902a03b2f62dca01939"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.032048 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:42.063771 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:42.065024 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:42.066545 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:42.068930 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:42.069121 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.069415 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:42.069571 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.111256 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.1:37437
I20260709 13:16:42.111524 10237 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.1:37437 every 8 connection(s)
I20260709 13:16:42.116577 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:42.124487 10242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:42.125640 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:42.128594 10245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:42.129277 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:42.130761 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:42.131006 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:42.131244 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783603002131224 us; error 0 us; skew 500 ppm
I20260709 13:16:42.131867 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:42.134343 32224 webserver.cc:533] Webserver started at http://127.31.120.2:36215/ using document root <none> and password file <none>
I20260709 13:16:42.134977 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:42.135232 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:42.135586 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:42.137223 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-1-root/instance:
uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.138409 10238 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42313
I20260709 13:16:42.138944 10238 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:42.139971 10238 heartbeater.cc:507] Master 127.31.120.62:42313 requested a full tablet report, sending...
I20260709 13:16:42.142598 10111 ts_manager.cc:194] Registered new tserver with Master: 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437)
I20260709 13:16:42.143550 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:42.144661 10111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39658
I20260709 13:16:42.147869 10250 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.148733 32224 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:16:42.149096 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-1-root
uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.149381 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:42.160993 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:42.162241 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:42.163792 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:42.166152 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:42.166404 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.166662 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:42.166859 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.211936 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.2:40809
I20260709 13:16:42.212019 10312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.2:40809 every 8 connection(s)
I20260709 13:16:42.216951 32224 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:42.225981 10316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:42.226926 10317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:42.229053 10313 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42313
W20260709 13:16:42.229300 10319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:42.229636 10313 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:42.230566 10313 heartbeater.cc:507] Master 127.31.120.62:42313 requested a full tablet report, sending...
I20260709 13:16:42.231205 32224 server_base.cc:1061] running on GCE node
I20260709 13:16:42.232373 32224 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:42.232625 32224 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:42.232827 32224 hybrid_clock.cc:648] HybridClock initialized: now 1783603002232811 us; error 0 us; skew 500 ppm
I20260709 13:16:42.233049 10111 ts_manager.cc:194] Registered new tserver with Master: 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:42.233572 32224 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:42.234951 10111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39670
I20260709 13:16:42.236812 32224 webserver.cc:533] Webserver started at http://127.31.120.3:44941/ using document root <none> and password file <none>
I20260709 13:16:42.237361 32224 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:42.237557 32224 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:42.237840 32224 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:42.238960 32224 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-2-root/instance:
uuid: "c97b79fc3e2546ee92b3896deb664b20"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.243324 32224 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:42.246819 10324 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.247849 32224 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:16:42.248152 32224 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-2-root
uuid: "c97b79fc3e2546ee92b3896deb664b20"
format_stamp: "Formatted at 2026-07-09 13:16:42 on dist-test-slave-pm98"
I20260709 13:16:42.248449 32224 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNSpHVS/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602715062314-32224-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:42.268859 32224 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:42.270056 32224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:42.271495 32224 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:42.273729 32224 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:42.273955 32224 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.274185 32224 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:42.274364 32224 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:42.315455 32224 rpc_server.cc:307] RPC server started. Bound to: 127.31.120.3:42141
I20260709 13:16:42.315541 10386 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.120.3:42141 every 8 connection(s)
I20260709 13:16:42.329247 10387 heartbeater.cc:344] Connected to a master server at 127.31.120.62:42313
I20260709 13:16:42.329658 10387 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:42.330400 10387 heartbeater.cc:507] Master 127.31.120.62:42313 requested a full tablet report, sending...
I20260709 13:16:42.332626 10111 ts_manager.cc:194] Registered new tserver with Master: c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:42.333595 32224 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014816043s
I20260709 13:16:42.334118 10111 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39676
I20260709 13:16:42.359186 10111 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39690:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:42.361903 10111 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:42.440698 10203 tablet_service.cc:1511] Processing CreateTablet for tablet 59a2f411ec384b428052c5dbf957e464 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:42.442233 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a2f411ec384b428052c5dbf957e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.444573 10202 tablet_service.cc:1511] Processing CreateTablet for tablet 6fee7a135f4a4253a72e695a15cf216a (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:42.446012 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee7a135f4a4253a72e695a15cf216a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.448673 10201 tablet_service.cc:1511] Processing CreateTablet for tablet 6a86931f91a949088cffe4d239de9f49 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:42.450162 10201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a86931f91a949088cffe4d239de9f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.452612 10200 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:42.454089 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.455461 10198 tablet_service.cc:1511] Processing CreateTablet for tablet e8bd4b8feb90401d9ad536672e687d90 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:42.456995 10198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bd4b8feb90401d9ad536672e687d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.453684 10199 tablet_service.cc:1511] Processing CreateTablet for tablet e3ff0a1b9b334e3287ba2887b852ba4f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:42.459473 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ff0a1b9b334e3287ba2887b852ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.464977 10197 tablet_service.cc:1511] Processing CreateTablet for tablet df5108da9a84462c80d89f0fad32ca6f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:42.466495 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5108da9a84462c80d89f0fad32ca6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.475555 10352 tablet_service.cc:1511] Processing CreateTablet for tablet 59a2f411ec384b428052c5dbf957e464 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:42.476243 10278 tablet_service.cc:1511] Processing CreateTablet for tablet 59a2f411ec384b428052c5dbf957e464 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:42.477140 10352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a2f411ec384b428052c5dbf957e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.477695 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a2f411ec384b428052c5dbf957e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.477594 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:42.479096 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.480849 10272 tablet_service.cc:1511] Processing CreateTablet for tablet df5108da9a84462c80d89f0fad32ca6f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:42.481679 10351 tablet_service.cc:1511] Processing CreateTablet for tablet 6fee7a135f4a4253a72e695a15cf216a (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:42.482045 10276 tablet_service.cc:1511] Processing CreateTablet for tablet 6a86931f91a949088cffe4d239de9f49 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:42.483544 10351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee7a135f4a4253a72e695a15cf216a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.484468 10275 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:42.484858 10270 tablet_service.cc:1511] Processing CreateTablet for tablet e92a28ff744a4d46b44d77f338a207fe (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:42.486264 10271 tablet_service.cc:1511] Processing CreateTablet for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:42.486768 10274 tablet_service.cc:1511] Processing CreateTablet for tablet e3ff0a1b9b334e3287ba2887b852ba4f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:42.482313 10272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5108da9a84462c80d89f0fad32ca6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.488739 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e92a28ff744a4d46b44d77f338a207fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.488996 10273 tablet_service.cc:1511] Processing CreateTablet for tablet e8bd4b8feb90401d9ad536672e687d90 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:42.489888 10271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.490377 10350 tablet_service.cc:1511] Processing CreateTablet for tablet 6a86931f91a949088cffe4d239de9f49 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:42.480849 10277 tablet_service.cc:1511] Processing CreateTablet for tablet 6fee7a135f4a4253a72e695a15cf216a (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:42.492224 10350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a86931f91a949088cffe4d239de9f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.494465 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee7a135f4a4253a72e695a15cf216a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.495779 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ff0a1b9b334e3287ba2887b852ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.495883 10202 tablet_service.cc:1511] Processing CreateTablet for tablet e92a28ff744a4d46b44d77f338a207fe (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:42.496930 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bd4b8feb90401d9ad536672e687d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.497189 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e92a28ff744a4d46b44d77f338a207fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.498101 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.499321 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a86931f91a949088cffe4d239de9f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.516750 10349 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:42.518244 10349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4b7ae8aa4c442eaf47c51d58dc7be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.531028 10348 tablet_service.cc:1511] Processing CreateTablet for tablet e3ff0a1b9b334e3287ba2887b852ba4f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:42.534462 10347 tablet_service.cc:1511] Processing CreateTablet for tablet e8bd4b8feb90401d9ad536672e687d90 (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:42.537170 10346 tablet_service.cc:1511] Processing CreateTablet for tablet df5108da9a84462c80d89f0fad32ca6f (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:42.540383 10344 tablet_service.cc:1511] Processing CreateTablet for tablet e92a28ff744a4d46b44d77f338a207fe (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:42.553725 10345 tablet_service.cc:1511] Processing CreateTablet for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c (DEFAULT_TABLE table=test-workload [id=106e05b51d9b41d59cfec363c35354a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:42.584182 10345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d68b6b0712c4812a4c3dd5a8c09c46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.588165 10348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ff0a1b9b334e3287ba2887b852ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.608451 10344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e92a28ff744a4d46b44d77f338a207fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.660241 10347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bd4b8feb90401d9ad536672e687d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.660699 10401 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.666348 10346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5108da9a84462c80d89f0fad32ca6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:42.678155 10401 tablet_bootstrap.cc:654] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.681061 10403 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:42.684237 10402 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.687840 10403 tablet_bootstrap.cc:654] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.690681 10402 tablet_bootstrap.cc:654] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.698276 10401 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:42.698755 10401 ts_tablet_manager.cc:1403] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.000s
I20260709 13:16:42.699857 10402 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.700309 10402 ts_tablet_manager.cc:1403] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.010s
I20260709 13:16:42.700495 10403 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:42.701018 10403 ts_tablet_manager.cc:1403] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.006s
I20260709 13:16:42.701726 10401 raft_consensus.cc:359] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.702502 10401 raft_consensus.cc:385] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.702601 10402 raft_consensus.cc:359] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.702838 10401 raft_consensus.cc:740] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.703290 10402 raft_consensus.cc:385] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.703639 10402 raft_consensus.cc:740] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.703953 10403 raft_consensus.cc:359] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.703956 10401 consensus_queue.cc:260] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.704300 10402 consensus_queue.cc:260] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.704849 10403 raft_consensus.cc:385] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.705219 10403 raft_consensus.cc:740] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.706113 10403 consensus_queue.cc:260] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.713413 10403 ts_tablet_manager.cc:1434] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.007s
I20260709 13:16:42.713476 10313 heartbeater.cc:499] Master 127.31.120.62:42313 was elected leader, sending a full tablet report...
I20260709 13:16:42.714339 10387 heartbeater.cc:499] Master 127.31.120.62:42313 was elected leader, sending a full tablet report...
I20260709 13:16:42.714403 10402 ts_tablet_manager.cc:1434] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.007s
I20260709 13:16:42.715970 10402 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.723340 10238 heartbeater.cc:499] Master 127.31.120.62:42313 was elected leader, sending a full tablet report...
I20260709 13:16:42.723961 10402 tablet_bootstrap.cc:654] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.724751 10401 ts_tablet_manager.cc:1434] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.010s
I20260709 13:16:42.725862 10401 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.728873 10403 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:42.733407 10401 tablet_bootstrap.cc:654] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.734773 10402 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.735371 10402 ts_tablet_manager.cc:1403] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:16:42.739189 10402 raft_consensus.cc:359] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.740259 10402 raft_consensus.cc:385] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.740398 10403 tablet_bootstrap.cc:654] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.740860 10402 raft_consensus.cc:740] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.742033 10402 consensus_queue.cc:260] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.744830 10402 ts_tablet_manager.cc:1434] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:42.746500 10402 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.746526 10401 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:42.747538 10401 ts_tablet_manager.cc:1403] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 13:16:42.750932 10401 raft_consensus.cc:359] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.751832 10401 raft_consensus.cc:385] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.752205 10401 raft_consensus.cc:740] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.753093 10401 consensus_queue.cc:260] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.753623 10408 raft_consensus.cc:493] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.754477 10408 raft_consensus.cc:515] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.780316 10403 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:42.781668 10401 ts_tablet_manager.cc:1434] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.034s	user 0.019s	sys 0.011s
I20260709 13:16:42.783197 10407 raft_consensus.cc:493] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.783519 10403 ts_tablet_manager.cc:1403] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.010s
I20260709 13:16:42.784142 10401 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.783836 10407 raft_consensus.cc:515] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.782305 10402 tablet_bootstrap.cc:654] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.786798 10407 leader_election.cc:290] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:42.782195 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
W20260709 13:16:42.790843 10325 leader_election.cc:343] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.793467 10401 tablet_bootstrap.cc:654] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.795972 10403 raft_consensus.cc:359] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.797854 10408 leader_election.cc:290] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:42.820498 10408 raft_consensus.cc:493] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.821313 10408 raft_consensus.cc:515] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.824249 10408 leader_election.cc:290] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:42.826454 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:42.829052 10403 raft_consensus.cc:385] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.829447 10403 raft_consensus.cc:740] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.830174 10403 consensus_queue.cc:260] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
W20260709 13:16:42.831558 10325 leader_election.cc:343] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.831794 10402 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.835705 10402 ts_tablet_manager.cc:1403] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.090s	user 0.033s	sys 0.049s
I20260709 13:16:42.836549 10403 ts_tablet_manager.cc:1434] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:16:42.838894 10402 raft_consensus.cc:359] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.840530 10403 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:42.840348 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:42.840947 10402 raft_consensus.cc:385] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.841317 10402 raft_consensus.cc:740] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.842103 10402 consensus_queue.cc:260] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.840510 10401 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
W20260709 13:16:42.842429 10176 leader_election.cc:343] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.844010 10401 ts_tablet_manager.cc:1403] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.060s	user 0.025s	sys 0.024s
I20260709 13:16:42.844352 10402 ts_tablet_manager.cc:1434] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:16:42.847958 10402 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.847944 10401 raft_consensus.cc:359] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.852336 10401 raft_consensus.cc:385] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.852974 10401 raft_consensus.cc:740] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.853677 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:42.854091 10403 tablet_bootstrap.cc:654] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.853844 10401 consensus_queue.cc:260] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.855897 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:42.856485 10402 tablet_bootstrap.cc:654] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.858939 10401 ts_tablet_manager.cc:1434] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 13:16:42.860064 10401 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.866194 10401 tablet_bootstrap.cc:654] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.873653 10403 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:42.874217 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:42.874341 10403 ts_tablet_manager.cc:1403] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.005s
W20260709 13:16:42.876742 10325 leader_election.cc:343] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.877364 10325 leader_election.cc:304] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:42.877795 10402 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.878285 10408 raft_consensus.cc:2749] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:42.878401 10402 ts_tablet_manager.cc:1403] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.014s
I20260709 13:16:42.878756 10401 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:42.879283 10401 ts_tablet_manager.cc:1403] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:16:42.878916 10403 raft_consensus.cc:359] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
W20260709 13:16:42.879789 10325 leader_election.cc:343] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.879812 10403 raft_consensus.cc:385] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.880641 10403 raft_consensus.cc:740] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.880612 10325 leader_election.cc:304] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:42.881696 10408 raft_consensus.cc:2749] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:42.881649 10402 raft_consensus.cc:359] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.881842 10403 consensus_queue.cc:260] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.882669 10402 raft_consensus.cc:385] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.883071 10402 raft_consensus.cc:740] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
W20260709 13:16:42.883592 10176 leader_election.cc:343] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.884135 10176 leader_election.cc:304] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57da9927cb614902a03b2f62dca01939; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:42.883900 10402 consensus_queue.cc:260] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.884418 10403 ts_tablet_manager.cc:1434] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:16:42.885496 10407 raft_consensus.cc:2749] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:42.886179 10403 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:42.882297 10401 raft_consensus.cc:359] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.886471 10402 ts_tablet_manager.cc:1434] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:16:42.886724 10401 raft_consensus.cc:385] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.887077 10401 raft_consensus.cc:740] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.887782 10402 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.894196 10402 tablet_bootstrap.cc:654] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.894502 10403 tablet_bootstrap.cc:654] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.887830 10401 consensus_queue.cc:260] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.897508 10401 ts_tablet_manager.cc:1434] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:16:42.898411 10401 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.899833 10402 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.900398 10402 ts_tablet_manager.cc:1403] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 13:16:42.903549 10402 raft_consensus.cc:359] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.904419 10402 raft_consensus.cc:385] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.904846 10401 tablet_bootstrap.cc:654] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.904794 10402 raft_consensus.cc:740] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.905851 10402 consensus_queue.cc:260] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.911293 10402 ts_tablet_manager.cc:1434] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:16:42.912532 10402 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.914186 10401 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:42.914798 10401 ts_tablet_manager.cc:1403] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:16:42.918023 10403 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:42.918036 10401 raft_consensus.cc:359] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.918768 10403 ts_tablet_manager.cc:1403] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.013s
I20260709 13:16:42.919206 10402 tablet_bootstrap.cc:654] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.919165 10401 raft_consensus.cc:385] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.919793 10401 raft_consensus.cc:740] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.920852 10401 consensus_queue.cc:260] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.921844 10403 raft_consensus.cc:359] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.922672 10403 raft_consensus.cc:385] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.923058 10403 raft_consensus.cc:740] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.924032 10403 consensus_queue.cc:260] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.923768 10401 ts_tablet_manager.cc:1434] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:16:42.926002 10401 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
W20260709 13:16:42.933377 10159 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:42.931852 10403 ts_tablet_manager.cc:1434] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:16:42.938791 10402 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.937704 10403 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:42.939522 10402 ts_tablet_manager.cc:1403] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.008s
I20260709 13:16:42.940572 10401 tablet_bootstrap.cc:654] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.943425 10402 raft_consensus.cc:359] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.944466 10402 raft_consensus.cc:385] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.944880 10402 raft_consensus.cc:740] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.946313 10402 consensus_queue.cc:260] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.947528 10403 tablet_bootstrap.cc:654] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.948855 10402 ts_tablet_manager.cc:1434] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:42.949002 10401 tablet_bootstrap.cc:492] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:42.949703 10401 ts_tablet_manager.cc:1403] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 13:16:42.950363 10402 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.953251 10401 raft_consensus.cc:359] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.954164 10401 raft_consensus.cc:385] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.954566 10401 raft_consensus.cc:740] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.955554 10401 consensus_queue.cc:260] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.958335 10402 tablet_bootstrap.cc:654] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.965675 10401 ts_tablet_manager.cc:1434] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:16:42.967350 10401 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:42.970677 10402 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:42.971526 10402 ts_tablet_manager.cc:1403] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.001s
I20260709 13:16:42.975029 10402 raft_consensus.cc:359] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.976101 10402 raft_consensus.cc:385] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.976526 10402 raft_consensus.cc:740] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:42.977502 10402 consensus_queue.cc:260] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.979239 10408 raft_consensus.cc:493] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.979727 10408 raft_consensus.cc:515] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.980127 10403 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:42.980309 10402 ts_tablet_manager.cc:1434] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:42.980813 10403 ts_tablet_manager.cc:1403] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.020s
I20260709 13:16:42.981433 10402 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:42.981765 10408 leader_election.cc:290] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:42.983239 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:42.983590 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:42.984205 10213 raft_consensus.cc:2468] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
W20260709 13:16:42.985045 10325 leader_election.cc:343] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:42.985951 10325 leader_election.cc:304] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:42.986466 10409 raft_consensus.cc:493] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.987174 10409 raft_consensus.cc:515] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:42.987129 10408 raft_consensus.cc:2804] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:42.988082 10408 raft_consensus.cc:493] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.988479 10408 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:42.989964 10409 leader_election.cc:290] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:42.992251 10402 tablet_bootstrap.cc:654] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.992326 10401 tablet_bootstrap.cc:654] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:42.993994 10403 raft_consensus.cc:359] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:42.997354 10403 raft_consensus.cc:385] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:42.998036 10403 raft_consensus.cc:740] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.001931 10403 consensus_queue.cc:260] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.004838 10403 ts_tablet_manager.cc:1434] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 13:16:43.005753 10401 tablet_bootstrap.cc:492] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:43.006261 10403 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:43.006415 10401 ts_tablet_manager.cc:1403] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.006s
I20260709 13:16:43.009953 10401 raft_consensus.cc:359] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.010811 10401 raft_consensus.cc:385] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.011166 10401 raft_consensus.cc:740] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.012019 10401 consensus_queue.cc:260] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.014609 10401 ts_tablet_manager.cc:1434] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:16:43.015223 10403 tablet_bootstrap.cc:654] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.017331 10402 tablet_bootstrap.cc:492] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:43.017938 10402 ts_tablet_manager.cc:1403] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.008s
I20260709 13:16:43.021600 10402 raft_consensus.cc:359] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.022773 10402 raft_consensus.cc:385] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.023144 10402 raft_consensus.cc:740] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.024106 10402 consensus_queue.cc:260] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.027446 10402 ts_tablet_manager.cc:1434] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:16:43.028971 10402 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:43.036669 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.037302 10401 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:43.037400 10362 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.039278 10251 leader_election.cc:304] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.040149 10409 raft_consensus.cc:2804] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.040518 10409 raft_consensus.cc:493] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.040840 10409 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.047976 10402 tablet_bootstrap.cc:654] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.049445 10408 raft_consensus.cc:515] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.051084 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.051826 10213 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.052103 10401 tablet_bootstrap.cc:654] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.052836 10408 leader_election.cc:290] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.053656 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.054168 10213 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.058259 10401 tablet_bootstrap.cc:492] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:43.058297 10402 tablet_bootstrap.cc:492] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:43.059159 10401 ts_tablet_manager.cc:1403] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.008s
I20260709 13:16:43.060545 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.061496 10213 raft_consensus.cc:2468] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
W20260709 13:16:43.061899 10325 leader_election.cc:343] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:43.049552 10409 raft_consensus.cc:515] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:43.063081 10325 leader_election.cc:304] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:43.063369 10401 raft_consensus.cc:359] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.064170 10409 leader_election.cc:290] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.064504 10401 raft_consensus.cc:385] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.064952 10401 raft_consensus.cc:740] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.065596 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.065794 10401 consensus_queue.cc:260] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.066238 10213 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.059211 10402 ts_tablet_manager.cc:1403] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.001s
I20260709 13:16:43.067157 10408 raft_consensus.cc:2804] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.067790 10403 tablet_bootstrap.cc:492] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:43.068385 10403 ts_tablet_manager.cc:1403] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.063s	user 0.035s	sys 0.021s
I20260709 13:16:43.068768 10408 raft_consensus.cc:697] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.069741 10408 consensus_queue.cc:237] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.069873 10402 raft_consensus.cc:359] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.070670 10402 raft_consensus.cc:385] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.070981 10402 raft_consensus.cc:740] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.071983 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.071911 10402 consensus_queue.cc:260] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.072626 10362 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.073938 10402 ts_tablet_manager.cc:1434] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:43.075981 10401 ts_tablet_manager.cc:1434] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:16:43.075857 10403 raft_consensus.cc:359] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.076774 10403 raft_consensus.cc:385] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.077111 10403 raft_consensus.cc:740] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.077406 10401 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:43.078040 10403 consensus_queue.cc:260] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.083338 10403 ts_tablet_manager.cc:1434] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:16:43.083429 10401 tablet_bootstrap.cc:654] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.084656 10403 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:43.094071 10403 tablet_bootstrap.cc:654] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.095633 10401 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:43.096246 10401 ts_tablet_manager.cc:1403] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:16:43.100047 10409 raft_consensus.cc:493] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.099843 10401 raft_consensus.cc:359] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.100610 10401 raft_consensus.cc:385] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.100620 10409 raft_consensus.cc:515] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.101120 10401 raft_consensus.cc:740] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.101996 10401 consensus_queue.cc:260] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.103066 10409 leader_election.cc:290] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.104738 10401 ts_tablet_manager.cc:1434] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:43.105793 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5108da9a84462c80d89f0fad32ca6f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.106415 10212 raft_consensus.cc:2468] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.107524 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5108da9a84462c80d89f0fad32ca6f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.107712 10251 leader_election.cc:304] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.108435 10361 raft_consensus.cc:2468] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.109372 10409 raft_consensus.cc:2804] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.109846 10409 raft_consensus.cc:493] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.110275 10409 raft_consensus.cc:3060] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.119549 10409 raft_consensus.cc:515] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.123080 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5108da9a84462c80d89f0fad32ca6f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.123802 10212 raft_consensus.cc:3060] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.124008 10403 tablet_bootstrap.cc:492] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:43.124761 10403 ts_tablet_manager.cc:1403] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.002s
I20260709 13:16:43.128337 10403 raft_consensus.cc:359] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.129366 10403 raft_consensus.cc:385] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.129763 10403 raft_consensus.cc:740] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.130697 10403 consensus_queue.cc:260] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.132771 10212 raft_consensus.cc:2468] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.136441 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5108da9a84462c80d89f0fad32ca6f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.137426 10361 raft_consensus.cc:3060] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.137792 10362 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.138185 10418 raft_consensus.cc:493] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.138818 10418 raft_consensus.cc:515] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.139635 10251 leader_election.cc:304] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.141330 10418 leader_election.cc:290] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.142694 10439 raft_consensus.cc:2804] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.142884 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.143368 10439 raft_consensus.cc:697] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:43.143743 10211 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:43.143961 10403 ts_tablet_manager.cc:1434] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.006s
I20260709 13:16:43.144371 10439 consensus_queue.cc:237] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:43.145432 10403 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:43.146292 10361 raft_consensus.cc:2468] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.147423 10409 leader_election.cc:290] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.147094 10325 leader_election.cc:304] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.136503 10213 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.148757 10418 raft_consensus.cc:2804] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.149217 10418 raft_consensus.cc:493] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.149668 10418 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.153928 10403 tablet_bootstrap.cc:654] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.158947 10251 leader_election.cc:304] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.160957 10440 raft_consensus.cc:493] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.161604 10440 raft_consensus.cc:515] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.163693 10439 raft_consensus.cc:2804] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.164223 10440 leader_election.cc:290] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.164389 10439 raft_consensus.cc:697] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:43.164690 10418 raft_consensus.cc:515] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.165865 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.166599 10403 tablet_bootstrap.cc:492] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:43.166528 10288 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:43.165622 10439 consensus_queue.cc:237] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.166800 10418 leader_election.cc:290] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.167114 10403 ts_tablet_manager.cc:1403] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:16:43.168419 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.169059 10213 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.169610 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.169979 10403 raft_consensus.cc:359] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.170260 10212 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.170677 10403 raft_consensus.cc:385] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.170977 10403 raft_consensus.cc:740] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.172042 10403 consensus_queue.cc:260] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.174098 10403 ts_tablet_manager.cc:1434] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:43.175006 10403 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:43.179770 10213 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:43.181423 10403 tablet_bootstrap.cc:654] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:43.181128 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.181830 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.182013 10362 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:43.181123 10325 leader_election.cc:304] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.182516 10288 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.183290 10408 raft_consensus.cc:2804] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.183513 10251 leader_election.cc:304] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.183945 10408 raft_consensus.cc:697] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.178578 10109 catalog_manager.cc:5697] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.185092 10408 consensus_queue.cc:237] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.187875 10440 raft_consensus.cc:2804] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.188544 10440 raft_consensus.cc:493] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.189013 10440 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.192837 10288 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:43.202551 10440 raft_consensus.cc:515] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.203753 10407 raft_consensus.cc:493] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.205217 10407 raft_consensus.cc:515] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.206426 10403 tablet_bootstrap.cc:492] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:43.206373 10408 raft_consensus.cc:493] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.207154 10403 ts_tablet_manager.cc:1403] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.012s
I20260709 13:16:43.207262 10407 leader_election.cc:290] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.207206 10408 raft_consensus.cc:515] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.209726 10440 leader_election.cc:290] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.210024 10408 leader_election.cc:290] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.210147 10403 raft_consensus.cc:359] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.211053 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.211791 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.211814 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.211864 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.218585 10361 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.212762 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.211407 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.219708 10212 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:43.218673 10288 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.221148 10408 raft_consensus.cc:493] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.221894 10176 leader_election.cc:304] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.224514 10108 catalog_manager.cc:5697] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.225880 10407 raft_consensus.cc:2804] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.227058 10407 raft_consensus.cc:493] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.218645 10213 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.228174 10407 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.229631 10361 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.211047 10403 raft_consensus.cc:385] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:43.227085 10325 leader_election.cc:304] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:43.230630 10403 raft_consensus.cc:740] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.222111 10408 raft_consensus.cc:515] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.231791 10403 consensus_queue.cc:260] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.218966 10362 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.233044 10418 raft_consensus.cc:2804] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.235154 10418 raft_consensus.cc:493] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.235757 10213 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:43.236083 10251 leader_election.cc:304] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.236873 10440 raft_consensus.cc:2804] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.236915 10403 ts_tablet_manager.cc:1434] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:16:43.235502 10418 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.237282 10440 raft_consensus.cc:697] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:43.240525 10440 consensus_queue.cc:237] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.243896 10408 leader_election.cc:290] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.246237 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.250126 10288 raft_consensus.cc:2468] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:43.247123 10418 raft_consensus.cc:515] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.251518 10325 leader_election.cc:304] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.248445 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.252682 10213 raft_consensus.cc:2468] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:43.252733 10418 leader_election.cc:290] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.254047 10418 raft_consensus.cc:2804] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.254448 10418 raft_consensus.cc:493] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.253860 10108 catalog_manager.cc:5697] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.254782 10418 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.253855 10407 raft_consensus.cc:515] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.262437 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.263302 10213 raft_consensus.cc:2393] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c97b79fc3e2546ee92b3896deb664b20 in current term 1: Already voted for candidate 57da9927cb614902a03b2f62dca01939 in this term.
I20260709 13:16:43.263332 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.264163 10288 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.264604 10407 leader_election.cc:290] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.265136 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.268729 10418 raft_consensus.cc:515] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.269084 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.269987 10362 raft_consensus.cc:2393] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 57da9927cb614902a03b2f62dca01939 in current term 1: Already voted for candidate c97b79fc3e2546ee92b3896deb664b20 in this term.
I20260709 13:16:43.270869 10418 leader_election.cc:290] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:43.272408 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.273155 10213 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.272501 10108 catalog_manager.cc:5697] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.276397 10288 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:43.277035 10109 catalog_manager.cc:5697] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.278393 10176 leader_election.cc:304] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57da9927cb614902a03b2f62dca01939; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:43.280251 10213 raft_consensus.cc:2468] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:43.280934 10325 leader_election.cc:304] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939
I20260709 13:16:43.281317 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.281989 10418 raft_consensus.cc:2804] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.282284 10288 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.282562 10325 leader_election.cc:304] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.279559 10407 raft_consensus.cc:2749] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:43.283767 10450 raft_consensus.cc:2804] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.284466 10450 raft_consensus.cc:697] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.282740 10418 raft_consensus.cc:697] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.285295 10450 consensus_queue.cc:237] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.286617 10418 consensus_queue.cc:237] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.293681 10288 raft_consensus.cc:2468] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:43.298164 10108 catalog_manager.cc:5697] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.299659 10108 catalog_manager.cc:5697] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.315706 10407 raft_consensus.cc:493] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.316365 10407 raft_consensus.cc:515] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.318208 10407 leader_election.cc:290] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.319010 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.319407 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.319738 10288 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.320046 10362 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.320907 10176 leader_election.cc:304] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.321651 10407 raft_consensus.cc:2804] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.322041 10407 raft_consensus.cc:493] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.322399 10407 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.327534 10407 raft_consensus.cc:515] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.329329 10407 leader_election.cc:290] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.330152 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.330262 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.330931 10288 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.331033 10362 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.336498 10288 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:43.336551 10362 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:43.337651 10176 leader_election.cc:304] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.338449 10407 raft_consensus.cc:2804] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.339514 10407 raft_consensus.cc:697] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:43.340292 10407 consensus_queue.cc:237] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.348007 10108 catalog_manager.cc:5697] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.420799 10440 raft_consensus.cc:493] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.421506 10440 raft_consensus.cc:515] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.423326 10440 leader_election.cc:290] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.424258 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:43.424499 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.425053 10213 raft_consensus.cc:2393] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate c97b79fc3e2546ee92b3896deb664b20 in this term.
I20260709 13:16:43.426157 10251 leader_election.cc:304] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:43.426800 10440 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.432147 10440 raft_consensus.cc:2749] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:43.474995 10418 consensus_queue.cc:1048] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.487547 10450 consensus_queue.cc:1048] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:43.508172 10407 raft_consensus.cc:493] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.508766 10407 raft_consensus.cc:515] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.510393 10407 leader_election.cc:290] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.511264 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:43.511510 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:43.511948 10288 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.512157 10362 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:43.512884 10176 leader_election.cc:304] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.513649 10407 raft_consensus.cc:2804] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:43.513936 10407 raft_consensus.cc:493] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.514202 10407 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.519166 10407 raft_consensus.cc:515] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.520727 10407 leader_election.cc:290] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.521546 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.521793 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.522166 10288 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.522446 10362 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:43.527273 10288 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:43.527653 10362 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:43.528489 10176 leader_election.cc:304] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.529233 10407 raft_consensus.cc:2804] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:43.529651 10407 raft_consensus.cc:697] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:43.530431 10407 consensus_queue.cc:237] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.537506 10109 catalog_manager.cc:5697] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.552182 10440 consensus_queue.cc:1048] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.564301 10409 consensus_queue.cc:1048] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.574908 10409 consensus_queue.cc:1048] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.587319 10440 consensus_queue.cc:1048] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:43.602564 10450 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.614122 10408 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.623322 10440 consensus_queue.cc:1048] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.635813 10440 consensus_queue.cc:1048] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.698285 10288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
mode: GRACEFUL
new_leader_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:56692
I20260709 13:16:43.698808 10288 raft_consensus.cc:606] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Received request to transfer leadership to TS 57da9927cb614902a03b2f62dca01939
I20260709 13:16:43.712211 10362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:43.712843 10362 raft_consensus.cc:606] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 57da9927cb614902a03b2f62dca01939
I20260709 13:16:43.718385 10362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:43.718921 10362 raft_consensus.cc:606] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 57da9927cb614902a03b2f62dca01939
I20260709 13:16:43.727571 10362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:43.728210 10362 raft_consensus.cc:606] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:43.742105 10213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:16:43.742722 10213 raft_consensus.cc:606] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:43.751674 10362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:43.752185 10362 raft_consensus.cc:606] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:43.757305 10287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
mode: GRACEFUL
new_leader_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:56692
I20260709 13:16:43.757943 10287 raft_consensus.cc:606] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Received request to transfer leadership to TS c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:43.758499 10408 consensus_queue.cc:1048] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.769200 10450 consensus_queue.cc:1048] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.770744 10418 consensus_queue.cc:1048] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.778471 10418 raft_consensus.cc:993] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:43.778966 10418 raft_consensus.cc:1081] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:43.783308 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56678
I20260709 13:16:43.783939 10288 raft_consensus.cc:493] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:43.784341 10288 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.786203 10450 consensus_queue.cc:1048] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.790632 10418 raft_consensus.cc:993] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:43.791625 10450 raft_consensus.cc:1081] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:43.791977 10288 raft_consensus.cc:515] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.793623 10212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:60716
I20260709 13:16:43.794463 10212 raft_consensus.cc:493] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:43.794883 10212 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.797827 10288 leader_election.cc:290] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.797955 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.799000 10362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d68b6b0712c4812a4c3dd5a8c09c46c" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.799642 10362 raft_consensus.cc:3055] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:43.800027 10362 raft_consensus.cc:740] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.800921 10362 consensus_queue.cc:260] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.802080 10362 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:43.803717 10450 consensus_queue.cc:1175] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "57da9927cb614902a03b2f62dca01939" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:16:43.803862 10212 raft_consensus.cc:515] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.807297 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:43.807703 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fee7a135f4a4253a72e695a15cf216a" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.807930 10288 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.808352 10361 raft_consensus.cc:3055] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:43.808689 10361 raft_consensus.cc:740] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:43.809628 10362 raft_consensus.cc:2468] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:43.809561 10361 consensus_queue.cc:260] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.810791 10361 raft_consensus.cc:3060] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.810932 10251 leader_election.cc:304] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939
I20260709 13:16:43.812309 10409 raft_consensus.cc:2804] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:43.812805 10409 raft_consensus.cc:697] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:43.813663 10409 consensus_queue.cc:237] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.814137 10407 consensus_queue.cc:1048] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.807978 10212 leader_election.cc:290] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.818231 10288 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:43.819698 10176 leader_election.cc:304] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:43.820765 10407 raft_consensus.cc:2804] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:43.821274 10407 raft_consensus.cc:697] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:43.822146 10407 consensus_queue.cc:237] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.822392 10108 catalog_manager.cc:5697] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.828660 10361 raft_consensus.cc:2468] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:43.832870 10111 catalog_manager.cc:5697] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.843865 10479 consensus_queue.cc:1048] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.855582 10478 raft_consensus.cc:993] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:43.856087 10479 raft_consensus.cc:1081] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:43.857666 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56662
I20260709 13:16:43.858302 10288 raft_consensus.cc:493] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:43.858724 10288 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.865258 10288 raft_consensus.cc:515] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.867514 10288 leader_election.cc:290] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:43.868424 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:43.868631 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bd4b8feb90401d9ad536672e687d90" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:43.869083 10212 raft_consensus.cc:3055] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:43.869243 10360 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.869428 10212 raft_consensus.cc:740] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:43.870280 10212 consensus_queue.cc:260] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.871280 10212 raft_consensus.cc:3060] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.874852 10360 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:43.876085 10251 leader_election.cc:304] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:43.876688 10212 raft_consensus.cc:2468] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:43.876792 10409 raft_consensus.cc:2804] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:43.877566 10409 raft_consensus.cc:697] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:43.878443 10409 consensus_queue.cc:237] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:43.885489 10111 catalog_manager.cc:5697] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from 57da9927cb614902a03b2f62dca01939 (127.31.120.1) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.925993 10479 consensus_queue.cc:1048] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:43.937492 10478 consensus_queue.cc:1048] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:44.018574 10450 raft_consensus.cc:993] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:44.019097 10408 raft_consensus.cc:1081] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:44.020910 10212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:60716
I20260709 13:16:44.021631 10212 raft_consensus.cc:493] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:44.021967 10212 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.027228 10212 raft_consensus.cc:515] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.028890 10212 leader_election.cc:290] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:44.029824 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:44.030117 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a86931f91a949088cffe4d239de9f49" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:44.030435 10288 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.030836 10360 raft_consensus.cc:3055] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.031141 10360 raft_consensus.cc:740] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:44.031889 10360 consensus_queue.cc:260] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.032841 10360 raft_consensus.cc:3060] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.036298 10288 raft_consensus.cc:2468] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:44.037549 10176 leader_election.cc:304] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:44.038257 10478 raft_consensus.cc:2804] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:44.038239 10360 raft_consensus.cc:2468] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:44.039106 10478 raft_consensus.cc:697] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:44.040025 10478 consensus_queue.cc:237] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.046828 10111 catalog_manager.cc:5697] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:44.072953 10440 raft_consensus.cc:993] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: : Instructing follower c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:44.073494 10440 raft_consensus.cc:1081] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Signalling peer c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:44.074344 10439 raft_consensus.cc:993] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: : Instructing follower 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:44.074860 10409 raft_consensus.cc:1081] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Signalling peer 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:44.075583 10360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:38280
I20260709 13:16:44.076251 10360 raft_consensus.cc:493] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:44.076395 10212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:60718
I20260709 13:16:44.076635 10360 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.077453 10212 raft_consensus.cc:493] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:44.077874 10212 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.084502 10360 raft_consensus.cc:515] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.085251 10212 raft_consensus.cc:515] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:44.086851 10360 leader_election.cc:290] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:44.088119 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:44.088255 10212 leader_election.cc:290] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Requested vote from peers c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:44.088742 10213 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.090687 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:44.091343 10288 raft_consensus.cc:3055] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.091697 10288 raft_consensus.cc:740] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:44.092146 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a2f411ec384b428052c5dbf957e464" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:44.092909 10360 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.092651 10288 consensus_queue.cc:260] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:44.093210 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e92a28ff744a4d46b44d77f338a207fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:44.094054 10287 raft_consensus.cc:3055] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.094451 10287 raft_consensus.cc:740] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:44.095235 10287 consensus_queue.cc:260] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.094050 10288 raft_consensus.cc:3060] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.096211 10213 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:44.096544 10287 raft_consensus.cc:3060] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.097625 10325 leader_election.cc:304] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:44.098639 10408 raft_consensus.cc:2804] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:44.099129 10408 raft_consensus.cc:697] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:44.100061 10408 consensus_queue.cc:237] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.100601 10360 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:44.101919 10176 leader_election.cc:304] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:44.102777 10478 raft_consensus.cc:2804] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:44.103189 10478 raft_consensus.cc:697] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:44.104084 10478 consensus_queue.cc:237] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } }
I20260709 13:16:44.104727 10287 raft_consensus.cc:2468] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:44.109768 10288 raft_consensus.cc:2468] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:44.110709 10111 catalog_manager.cc:5697] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 1 to 2, leader changed from 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2) to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:44.112353 10109 catalog_manager.cc:5697] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 1 to 2, leader changed from 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2) to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:44.129678 10450 raft_consensus.cc:993] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:44.130164 10408 raft_consensus.cc:1081] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:44.131706 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56678
I20260709 13:16:44.132231 10288 raft_consensus.cc:493] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:44.132606 10288 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.137907 10288 raft_consensus.cc:515] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.139580 10288 leader_election.cc:290] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:44.140308 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:44.140728 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:44.141047 10213 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.141379 10360 raft_consensus.cc:3055] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.141705 10360 raft_consensus.cc:740] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:44.142524 10360 consensus_queue.cc:260] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.143774 10360 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.148279 10213 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:44.149664 10251 leader_election.cc:304] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:44.150419 10409 raft_consensus.cc:2804] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:44.150741 10360 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:44.150935 10409 raft_consensus.cc:697] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:44.151880 10409 consensus_queue.cc:237] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:44.158385 10109 catalog_manager.cc:5697] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:44.289919 10360 raft_consensus.cc:1275] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.291538 10439 consensus_queue.cc:1048] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.292968 10360 raft_consensus.cc:1275] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.294368 10439 consensus_queue.cc:1048] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.300443 10213 raft_consensus.cc:3060] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.305661 10212 raft_consensus.cc:1275] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.310114 10288 raft_consensus.cc:1275] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.310695 10213 raft_consensus.cc:1275] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.311832 10407 consensus_queue.cc:1048] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.315121 10440 consensus_queue.cc:1048] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.316730 10361 raft_consensus.cc:1275] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.317363 10483 consensus_queue.cc:1048] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:44.318432 10478 consensus_queue.cc:1048] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.453625 10361 raft_consensus.cc:1275] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.454527 10288 raft_consensus.cc:1275] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.454872 10479 consensus_queue.cc:1048] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.456101 10407 consensus_queue.cc:1048] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.489013 10288 raft_consensus.cc:1275] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.490653 10479 consensus_queue.cc:1048] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.491068 10288 raft_consensus.cc:1275] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.492530 10408 consensus_queue.cc:1048] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.499859 10361 raft_consensus.cc:1275] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.503042 10212 raft_consensus.cc:1275] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.504582 10479 consensus_queue.cc:1048] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.508392 10450 consensus_queue.cc:1048] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.599545 10212 raft_consensus.cc:1275] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.600805 10409 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:44.608681 10361 raft_consensus.cc:1275] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:44.611106 10483 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.790659 10111 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46884:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:46.794245 10111 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:46.846405 10201 tablet_service.cc:1511] Processing CreateTablet for tablet f0ffef919b39443b8acad51477e127d2 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:46.846872 10276 tablet_service.cc:1511] Processing CreateTablet for tablet f0ffef919b39443b8acad51477e127d2 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:46.848472 10201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ffef919b39443b8acad51477e127d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.848702 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ffef919b39443b8acad51477e127d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.851913 10347 tablet_service.cc:1511] Processing CreateTablet for tablet f0ffef919b39443b8acad51477e127d2 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:46.854786 10203 tablet_service.cc:1511] Processing CreateTablet for tablet 0e7e204958a642bba8e4842f16b99246 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:46.856582 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e7e204958a642bba8e4842f16b99246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.858071 10347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0ffef919b39443b8acad51477e127d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.867981 10273 tablet_service.cc:1511] Processing CreateTablet for tablet 0e7e204958a642bba8e4842f16b99246 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:46.868496 10277 tablet_service.cc:1511] Processing CreateTablet for tablet 305c3498123f466ca2cee7964b6fcd9f (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:46.869613 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e7e204958a642bba8e4842f16b99246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.869292 10348 tablet_service.cc:1511] Processing CreateTablet for tablet 0e7e204958a642bba8e4842f16b99246 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:46.871068 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305c3498123f466ca2cee7964b6fcd9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.871318 10348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e7e204958a642bba8e4842f16b99246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.880271 10345 tablet_service.cc:1511] Processing CreateTablet for tablet 305c3498123f466ca2cee7964b6fcd9f (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:46.881851 10345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305c3498123f466ca2cee7964b6fcd9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.883176 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce140755ec84b65a672c00aff4e8f4c (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:46.884737 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce140755ec84b65a672c00aff4e8f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.884589 10197 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce140755ec84b65a672c00aff4e8f4c (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:46.886032 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce140755ec84b65a672c00aff4e8f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.876017 10200 tablet_service.cc:1511] Processing CreateTablet for tablet 305c3498123f466ca2cee7964b6fcd9f (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:46.888582 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305c3498123f466ca2cee7964b6fcd9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.893594 10275 tablet_service.cc:1511] Processing CreateTablet for tablet f9240a430c924d8788abe52364490dc3 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:46.897501 10346 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce140755ec84b65a672c00aff4e8f4c (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:46.898972 10346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce140755ec84b65a672c00aff4e8f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.901759 10352 tablet_service.cc:1511] Processing CreateTablet for tablet f9240a430c924d8788abe52364490dc3 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:46.903298 10352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9240a430c924d8788abe52364490dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.905551 10196 tablet_service.cc:1511] Processing CreateTablet for tablet f9240a430c924d8788abe52364490dc3 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:46.906952 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9240a430c924d8788abe52364490dc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.895501 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9240a430c924d8788abe52364490dc3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:46.944309 10176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.2:40809, user_credentials={real_user=slave}} blocked reactor thread for 80526.4us
I20260709 13:16:46.965160 10199 tablet_service.cc:1511] Processing CreateTablet for tablet fd35d27a47e64b6f9ab031abb6739943 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:46.966941 10344 tablet_service.cc:1511] Processing CreateTablet for tablet ed1b098347994e6094d96b8eef741247 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:46.967495 10198 tablet_service.cc:1511] Processing CreateTablet for tablet ed1b098347994e6094d96b8eef741247 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:46.968680 10344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1b098347994e6094d96b8eef741247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.968899 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:46.969527 10202 tablet_service.cc:1511] Processing CreateTablet for tablet 17ff154cdf8c460a9b315cb0c88d3007 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:46.971585 10350 tablet_service.cc:1511] Processing CreateTablet for tablet fd35d27a47e64b6f9ab031abb6739943 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:46.972990 10350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd35d27a47e64b6f9ab031abb6739943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.976929 10351 tablet_service.cc:1511] Processing CreateTablet for tablet 17ff154cdf8c460a9b315cb0c88d3007 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:46.978396 10351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ff154cdf8c460a9b315cb0c88d3007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.980929 10349 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:46.982411 10349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:46.985311 10159 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:47.012580 10499 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.034858 10277 tablet_service.cc:1511] Processing CreateTablet for tablet ed1b098347994e6094d96b8eef741247 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:47.036552 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1b098347994e6094d96b8eef741247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.036442 10275 tablet_service.cc:1511] Processing CreateTablet for tablet fd35d27a47e64b6f9ab031abb6739943 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:47.037899 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd35d27a47e64b6f9ab031abb6739943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.035470 10501 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.045879 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:47.047210 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.047148 10276 tablet_service.cc:1511] Processing CreateTablet for tablet 17ff154cdf8c460a9b315cb0c88d3007 (DEFAULT_TABLE table=soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:47.048537 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ff154cdf8c460a9b315cb0c88d3007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.066203 10501 tablet_bootstrap.cc:654] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.067391 10198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1b098347994e6094d96b8eef741247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.073331 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0f0cd93c934b6eb96bdca6827fe1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.081192 10502 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.086234 10499 tablet_bootstrap.cc:654] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.107157 10502 tablet_bootstrap.cc:654] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.108654 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd35d27a47e64b6f9ab031abb6739943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.112732 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ff154cdf8c460a9b315cb0c88d3007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:47.166549 10501 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.167235 10501 ts_tablet_manager.cc:1403] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.132s	user 0.025s	sys 0.026s
I20260709 13:16:47.168062 10502 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.168699 10502 ts_tablet_manager.cc:1403] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.088s	user 0.010s	sys 0.020s
I20260709 13:16:47.170049 10499 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.170584 10499 ts_tablet_manager.cc:1403] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.032s
I20260709 13:16:47.170640 10501 raft_consensus.cc:359] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.171682 10501 raft_consensus.cc:385] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.172058 10501 raft_consensus.cc:740] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.171885 10502 raft_consensus.cc:359] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.172714 10502 raft_consensus.cc:385] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.173018 10502 raft_consensus.cc:740] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.173082 10501 consensus_queue.cc:260] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.173825 10499 raft_consensus.cc:359] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.173949 10502 consensus_queue.cc:260] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.174710 10499 raft_consensus.cc:385] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.175057 10499 raft_consensus.cc:740] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.175855 10499 consensus_queue.cc:260] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.176481 10502 ts_tablet_manager.cc:1434] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
W20260709 13:16:47.176378 10239 tablet.cc:2406] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:47.177835 10502 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.178547 10499 ts_tablet_manager.cc:1434] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:47.183038 10499 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.184957 10502 tablet_bootstrap.cc:654] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.189738 10499 tablet_bootstrap.cc:654] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.202437 10501 ts_tablet_manager.cc:1434] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:16:47.204131 10501 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.224567 10501 tablet_bootstrap.cc:654] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.225998 10502 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.225984 10499 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.226670 10502 ts_tablet_manager.cc:1403] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.049s	user 0.021s	sys 0.026s
I20260709 13:16:47.226681 10499 ts_tablet_manager.cc:1403] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.020s
I20260709 13:16:47.229465 10501 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.229972 10501 ts_tablet_manager.cc:1403] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.001s
I20260709 13:16:47.230192 10502 raft_consensus.cc:359] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.230404 10499 raft_consensus.cc:359] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.231117 10502 raft_consensus.cc:385] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.231166 10499 raft_consensus.cc:385] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.231566 10502 raft_consensus.cc:740] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.231648 10499 raft_consensus.cc:740] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.232508 10502 consensus_queue.cc:260] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.232849 10499 consensus_queue.cc:260] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.232873 10501 raft_consensus.cc:359] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.233909 10501 raft_consensus.cc:385] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.234283 10501 raft_consensus.cc:740] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.234930 10502 ts_tablet_manager.cc:1434] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:16:47.235198 10501 consensus_queue.cc:260] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.235960 10502 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.239784 10501 ts_tablet_manager.cc:1434] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:16:47.240912 10501 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.249116 10499 ts_tablet_manager.cc:1434] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:16:47.249580 10501 tablet_bootstrap.cc:654] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.250483 10499 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.253185 10502 tablet_bootstrap.cc:654] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.255573 10501 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.262846 10499 tablet_bootstrap.cc:654] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.266505 10502 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.267091 10502 ts_tablet_manager.cc:1403] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.005s
I20260709 13:16:47.256150 10501 ts_tablet_manager.cc:1403] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:16:47.271355 10502 raft_consensus.cc:359] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.271435 10501 raft_consensus.cc:359] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.272478 10502 raft_consensus.cc:385] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.272563 10501 raft_consensus.cc:385] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.272871 10502 raft_consensus.cc:740] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.273041 10501 raft_consensus.cc:740] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.274256 10499 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.273861 10502 consensus_queue.cc:260] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.273859 10501 consensus_queue.cc:260] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.274797 10499 ts_tablet_manager.cc:1403] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.010s
I20260709 13:16:47.276466 10501 ts_tablet_manager.cc:1434] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:47.277647 10501 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.278703 10499 raft_consensus.cc:359] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.279552 10499 raft_consensus.cc:385] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.279888 10499 raft_consensus.cc:740] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.280680 10499 consensus_queue.cc:260] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.284219 10499 ts_tablet_manager.cc:1434] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.010s
I20260709 13:16:47.286525 10502 ts_tablet_manager.cc:1434] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:16:47.287750 10502 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.292608 10499 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.298043 10502 tablet_bootstrap.cc:654] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.301208 10499 tablet_bootstrap.cc:654] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.303136 10501 tablet_bootstrap.cc:654] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.320304 10501 tablet_bootstrap.cc:492] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.320806 10501 ts_tablet_manager.cc:1403] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.007s
I20260709 13:16:47.321683 10499 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.321995 10502 tablet_bootstrap.cc:492] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.322474 10502 ts_tablet_manager.cc:1403] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.015s
I20260709 13:16:47.322353 10499 ts_tablet_manager.cc:1403] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.007s
I20260709 13:16:47.323239 10501 raft_consensus.cc:359] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.324095 10501 raft_consensus.cc:385] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.324442 10501 raft_consensus.cc:740] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.325353 10502 raft_consensus.cc:359] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.326064 10502 raft_consensus.cc:385] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.325364 10501 consensus_queue.cc:260] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.326325 10502 raft_consensus.cc:740] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.326872 10499 raft_consensus.cc:359] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.327073 10502 consensus_queue.cc:260] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.327661 10499 raft_consensus.cc:385] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.328064 10499 raft_consensus.cc:740] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.328766 10499 consensus_queue.cc:260] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.329459 10502 ts_tablet_manager.cc:1434] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:47.330515 10502 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.331023 10499 ts_tablet_manager.cc:1434] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.006s
I20260709 13:16:47.332055 10499 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.338047 10502 tablet_bootstrap.cc:654] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.338681 10500 raft_consensus.cc:493] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.339226 10500 raft_consensus.cc:515] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.340405 10499 tablet_bootstrap.cc:654] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.342978 10500 leader_election.cc:290] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.348171 10408 raft_consensus.cc:493] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.348773 10408 raft_consensus.cc:515] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.349964 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.357200 10500 raft_consensus.cc:493] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.351565 10408 leader_election.cc:290] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.352631 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.361380 10500 raft_consensus.cc:515] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.359943 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.364106 10500 leader_election.cc:290] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.364279 10212 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
W20260709 13:16:47.356702 10251 leader_election.cc:343] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.365895 10325 leader_election.cc:304] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.367775 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.368007 10408 raft_consensus.cc:2804] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.366072 10501 ts_tablet_manager.cc:1434] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.002s
I20260709 13:16:47.366951 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.368765 10502 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.368984 10287 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.369633 10502 ts_tablet_manager.cc:1403] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.004s
I20260709 13:16:47.369977 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.370674 10212 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:47.371320 10501 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
W20260709 13:16:47.369235 10251 leader_election.cc:343] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.371971 10251 leader_election.cc:304] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:47.372855 10500 raft_consensus.cc:2749] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:47.374434 10502 raft_consensus.cc:359] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.375064 10251 leader_election.cc:304] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.368549 10361 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:47.375497 10502 raft_consensus.cc:385] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.376013 10502 raft_consensus.cc:740] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.377125 10502 consensus_queue.cc:260] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.379485 10408 raft_consensus.cc:493] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.380017 10408 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.380383 10501 tablet_bootstrap.cc:654] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.381062 10499 tablet_bootstrap.cc:492] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.381587 10499 ts_tablet_manager.cc:1403] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.050s	user 0.031s	sys 0.008s
I20260709 13:16:47.383405 10502 ts_tablet_manager.cc:1434] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:16:47.384577 10502 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.384809 10499 raft_consensus.cc:359] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.385607 10499 raft_consensus.cc:385] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.385953 10499 raft_consensus.cc:740] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.386723 10499 consensus_queue.cc:260] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.389384 10499 ts_tablet_manager.cc:1434] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:16:47.390450 10499 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.392220 10502 tablet_bootstrap.cc:654] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.396140 10500 raft_consensus.cc:2804] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.396723 10500 raft_consensus.cc:493] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.397126 10500 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.398164 10499 tablet_bootstrap.cc:654] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.400063 10483 raft_consensus.cc:493] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.400568 10483 raft_consensus.cc:515] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.402637 10483 leader_election.cc:290] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.404189 10502 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.404728 10502 ts_tablet_manager.cc:1403] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 13:16:47.408125 10502 raft_consensus.cc:359] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.409124 10502 raft_consensus.cc:385] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.409317 10518 raft_consensus.cc:493] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.409574 10502 raft_consensus.cc:740] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.409901 10518 raft_consensus.cc:515] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.410817 10502 consensus_queue.cc:260] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.412077 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9240a430c924d8788abe52364490dc3" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.412837 10212 raft_consensus.cc:2468] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:47.413209 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.413895 10213 raft_consensus.cc:2468] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.415176 10325 leader_election.cc:304] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.416440 10499 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.416994 10499 ts_tablet_manager.cc:1403] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.007s
I20260709 13:16:47.417222 10502 ts_tablet_manager.cc:1434] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:16:47.416860 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9240a430c924d8788abe52364490dc3" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.418501 10502 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.420374 10499 raft_consensus.cc:359] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.421109 10499 raft_consensus.cc:385] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.421417 10499 raft_consensus.cc:740] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.422401 10499 consensus_queue.cc:260] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.424774 10499 ts_tablet_manager.cc:1434] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:16:47.425763 10499 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.429914 10502 tablet_bootstrap.cc:654] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.431528 10483 raft_consensus.cc:493] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.432140 10483 raft_consensus.cc:515] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.432793 10499 tablet_bootstrap.cc:654] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.428081 10408 raft_consensus.cc:515] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
W20260709 13:16:47.431735 10251 leader_election.cc:343] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.434480 10483 leader_election.cc:290] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.434691 10251 leader_election.cc:304] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:47.447223 10518 leader_election.cc:290] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.447918 10483 raft_consensus.cc:2804] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.450577 10483 raft_consensus.cc:493] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.448210 10521 raft_consensus.cc:2804] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.452029 10521 raft_consensus.cc:493] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.448848 10500 raft_consensus.cc:515] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.452375 10521 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.453431 10408 leader_election.cc:290] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.453833 10500 leader_election.cc:290] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.454936 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.455250 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.459743 10213 raft_consensus.cc:2468] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:47.451560 10483 raft_consensus.cc:3060] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.462365 10478 raft_consensus.cc:493] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.462960 10478 raft_consensus.cc:515] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.467227 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.469805 10502 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.470126 10361 raft_consensus.cc:2393] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate c97b79fc3e2546ee92b3896deb664b20 in this term.
I20260709 13:16:47.470504 10502 ts_tablet_manager.cc:1403] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.052s	user 0.032s	sys 0.011s
I20260709 13:16:47.471271 10499 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.471944 10499 ts_tablet_manager.cc:1403] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.003s
I20260709 13:16:47.473089 10478 leader_election.cc:290] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.474045 10502 raft_consensus.cc:359] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.474812 10502 raft_consensus.cc:385] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.475099 10502 raft_consensus.cc:740] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.466717 10251 leader_election.cc:304] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.469815 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.475229 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ffef919b39443b8acad51477e127d2" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.480518 10213 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.480710 10287 raft_consensus.cc:2468] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.481634 10502 consensus_queue.cc:260] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.482417 10176 leader_election.cc:304] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.480681 10521 raft_consensus.cc:515] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.481654 10501 tablet_bootstrap.cc:492] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.485180 10500 raft_consensus.cc:2804] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.485610 10500 raft_consensus.cc:493] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.485649 10501 ts_tablet_manager.cc:1403] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.004s
I20260709 13:16:47.485996 10500 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.487234 10478 raft_consensus.cc:2804] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.487717 10478 raft_consensus.cc:493] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.488078 10478 raft_consensus.cc:3060] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.489385 10501 raft_consensus.cc:359] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.490011 10213 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:47.490293 10501 raft_consensus.cc:385] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.490775 10501 raft_consensus.cc:740] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.491590 10501 consensus_queue.cc:260] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.483246 10483 raft_consensus.cc:515] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.493244 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ffef919b39443b8acad51477e127d2" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.493935 10361 raft_consensus.cc:2468] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.494477 10500 raft_consensus.cc:515] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.495267 10483 leader_election.cc:290] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.496503 10502 ts_tablet_manager.cc:1434] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:16:47.497704 10502 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.498479 10501 ts_tablet_manager.cc:1434] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:16:47.499851 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9240a430c924d8788abe52364490dc3" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.500658 10212 raft_consensus.cc:3060] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.475188 10499 raft_consensus.cc:359] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.502089 10499 raft_consensus.cc:385] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.502431 10499 raft_consensus.cc:740] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.503489 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.504467 10287 raft_consensus.cc:2393] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c97b79fc3e2546ee92b3896deb664b20 in current term 1: Already voted for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in this term.
I20260709 13:16:47.508054 10251 leader_election.cc:304] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:47.511088 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.511729 10211 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.512204 10483 raft_consensus.cc:2804] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.512771 10483 raft_consensus.cc:697] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:47.513486 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.514245 10483 consensus_queue.cc:237] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.515657 10360 raft_consensus.cc:2393] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate c97b79fc3e2546ee92b3896deb664b20 in this term.
I20260709 13:16:47.516570 10521 leader_election.cc:290] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.505249 10502 tablet_bootstrap.cc:654] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.505666 10500 leader_election.cc:290] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.505913 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.503247 10499 consensus_queue.cc:260] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.508029 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9240a430c924d8788abe52364490dc3" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.522980 10213 raft_consensus.cc:2393] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c97b79fc3e2546ee92b3896deb664b20 in current term 1: Already voted for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in this term.
I20260709 13:16:47.523654 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.522982 10361 raft_consensus.cc:3060] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.524473 10499 ts_tablet_manager.cc:1434] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 13:16:47.525549 10499 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.525660 10501 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.527041 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.527958 10360 raft_consensus.cc:2393] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate c97b79fc3e2546ee92b3896deb664b20 in this term.
I20260709 13:16:47.521451 10478 raft_consensus.cc:515] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.532198 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.532459 10361 raft_consensus.cc:2468] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:47.532979 10287 raft_consensus.cc:2393] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c97b79fc3e2546ee92b3896deb664b20 in current term 1: Already voted for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in this term.
W20260709 13:16:47.533620 10176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.31.120.2:40809, user_credentials={real_user=slave}} blocked reactor thread for 51307.5us
I20260709 13:16:47.534806 10251 leader_election.cc:304] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.532742 10211 raft_consensus.cc:2468] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:47.539477 10500 raft_consensus.cc:2804] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.540086 10500 raft_consensus.cc:697] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:47.540870 10525 raft_consensus.cc:493] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.541088 10500 consensus_queue.cc:237] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.541762 10525 raft_consensus.cc:515] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.544905 10478 leader_election.cc:290] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.546068 10502 tablet_bootstrap.cc:492] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.546568 10502 ts_tablet_manager.cc:1403] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 13:16:47.546808 10525 leader_election.cc:290] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.550154 10502 raft_consensus.cc:359] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.551811 10502 raft_consensus.cc:385] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.552166 10502 raft_consensus.cc:740] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.552991 10502 consensus_queue.cc:260] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.536520 10212 raft_consensus.cc:2468] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 1.
I20260709 13:16:47.555080 10499 tablet_bootstrap.cc:654] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.555279 10251 leader_election.cc:304] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:47.556476 10502 ts_tablet_manager.cc:1434] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:16:47.556485 10500 raft_consensus.cc:2804] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.557482 10502 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: Bootstrap starting.
I20260709 13:16:47.537634 10325 leader_election.cc:304] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:47.557648 10500 raft_consensus.cc:697] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:47.555635 10108 catalog_manager.cc:5697] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.561066 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ffef919b39443b8acad51477e127d2" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.563196 10287 raft_consensus.cc:3060] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.558115 10501 tablet_bootstrap.cc:654] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.564280 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.567308 10521 raft_consensus.cc:2749] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:47.568471 10502 tablet_bootstrap.cc:654] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.559536 10500 consensus_queue.cc:237] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
W20260709 13:16:47.577023 10176 leader_election.cc:343] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.578824 10325 leader_election.cc:304] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:47.581755 10499 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.581542 10287 raft_consensus.cc:2468] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.582361 10499 ts_tablet_manager.cc:1403] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.006s
I20260709 13:16:47.585178 10521 raft_consensus.cc:2749] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:47.590101 10408 raft_consensus.cc:493] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.583982 10176 leader_election.cc:304] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.590672 10501 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.591378 10501 ts_tablet_manager.cc:1403] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.010s
I20260709 13:16:47.590766 10408 raft_consensus.cc:515] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.591481 10529 raft_consensus.cc:2804] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.592198 10529 raft_consensus.cc:697] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:47.593060 10529 consensus_queue.cc:237] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.593926 10408 leader_election.cc:290] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.594774 10501 raft_consensus.cc:359] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.595561 10501 raft_consensus.cc:385] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.595932 10501 raft_consensus.cc:740] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.596771 10502 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: No bootstrap required, opened a new log
I20260709 13:16:47.597899 10502 ts_tablet_manager.cc:1403] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.004s
I20260709 13:16:47.596623 10501 consensus_queue.cc:260] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.601130 10499 raft_consensus.cc:359] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.601624 10502 raft_consensus.cc:359] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.602502 10502 raft_consensus.cc:385] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.602743 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0ffef919b39443b8acad51477e127d2" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.603258 10502 raft_consensus.cc:740] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.603400 10361 raft_consensus.cc:3060] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.604236 10501 ts_tablet_manager.cc:1434] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:16:47.604140 10502 consensus_queue.cc:260] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.605754 10501 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.606462 10502 ts_tablet_manager.cc:1434] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:47.602149 10499 raft_consensus.cc:385] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.607507 10499 raft_consensus.cc:740] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.608345 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.608345 10499 consensus_queue.cc:260] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.609212 10360 raft_consensus.cc:2468] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.610909 10499 ts_tablet_manager.cc:1434] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:16:47.612000 10499 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: Bootstrap starting.
I20260709 13:16:47.614450 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.615234 10212 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.618288 10499 tablet_bootstrap.cc:654] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.619637 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.620419 10287 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.626068 10176 leader_election.cc:304] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:47.627342 10325 leader_election.cc:304] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.627720 10361 raft_consensus.cc:2468] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.628103 10501 tablet_bootstrap.cc:654] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.628279 10408 raft_consensus.cc:2804] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.628739 10408 raft_consensus.cc:493] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.629159 10408 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.628593 10478 raft_consensus.cc:2804] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.633183 10478 raft_consensus.cc:493] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.633688 10478 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.644209 10529 raft_consensus.cc:493] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.644748 10499 tablet_bootstrap.cc:492] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: No bootstrap required, opened a new log
I20260709 13:16:47.644874 10529 raft_consensus.cc:515] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.644340 10111 catalog_manager.cc:5697] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.645582 10408 raft_consensus.cc:515] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.647904 10408 leader_election.cc:290] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.648279 10499 ts_tablet_manager.cc:1403] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.002s
I20260709 13:16:47.648790 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.649447 10212 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.649863 10529 leader_election.cc:290] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.649575 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.653648 10287 raft_consensus.cc:2468] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.654981 10176 leader_election.cc:304] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.657922 10529 raft_consensus.cc:2804] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.658282 10529 raft_consensus.cc:493] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.658610 10529 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.664707 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.664826 10499 raft_consensus.cc:359] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.665802 10499 raft_consensus.cc:385] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.665453 10287 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.666191 10499 raft_consensus.cc:740] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.666170 10212 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:47.667595 10499 consensus_queue.cc:260] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.671257 10529 raft_consensus.cc:515] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.672869 10499 ts_tablet_manager.cc:1434] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:16:47.673362 10529 leader_election.cc:290] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.675400 10287 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:47.676909 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.676987 10325 leader_election.cc:304] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.678285 10408 raft_consensus.cc:2804] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.678536 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.679154 10478 raft_consensus.cc:515] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.679711 10408 raft_consensus.cc:697] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:47.680505 10408 consensus_queue.cc:237] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.681524 10478 leader_election.cc:290] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.681881 10483 raft_consensus.cc:493] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.679215 10288 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.682372 10483 raft_consensus.cc:515] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.685019 10483 leader_election.cc:290] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.687034 10501 tablet_bootstrap.cc:492] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.686908 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.686908 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.688007 10360 raft_consensus.cc:2468] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 0.
I20260709 13:16:47.688002 10501 ts_tablet_manager.cc:1403] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.005s
I20260709 13:16:47.688226 10212 raft_consensus.cc:2393] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate 57da9927cb614902a03b2f62dca01939 in this term.
I20260709 13:16:47.689329 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.689340 10251 leader_election.cc:304] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
W20260709 13:16:47.678557 10176 leader_election.cc:343] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.690658 10483 raft_consensus.cc:2804] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.691144 10483 raft_consensus.cc:493] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.691547 10483 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.692507 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.692269 10501 raft_consensus.cc:359] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.693188 10501 raft_consensus.cc:385] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.693558 10501 raft_consensus.cc:740] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.694351 10501 consensus_queue.cc:260] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.696755 10501 ts_tablet_manager.cc:1434] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:16:47.697355 10288 raft_consensus.cc:2468] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.697876 10501 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.702363 10483 raft_consensus.cc:515] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.703399 10287 raft_consensus.cc:2393] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 57da9927cb614902a03b2f62dca01939 in current term 1: Already voted for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in this term.
I20260709 13:16:47.703946 10176 leader_election.cc:304] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.704996 10501 tablet_bootstrap.cc:654] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.705922 10478 raft_consensus.cc:2804] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:16:47.706321 10176 leader_election.cc:343] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:47.706566 10478 raft_consensus.cc:697] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:47.707439 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.707432 10478 consensus_queue.cc:237] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.708212 10361 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.707876 10108 catalog_manager.cc:5697] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:47.713544 10110 catalog_manager.cc:5697] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.716282 10110 catalog_manager.cc:5697] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 1 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.718868 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.719693 10212 raft_consensus.cc:2393] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c58a29b278c42f7a3dab0d07dd52a6d in current term 1: Already voted for candidate 57da9927cb614902a03b2f62dca01939 in this term.
I20260709 13:16:47.721946 10483 leader_election.cc:290] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.722694 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.724156 10251 leader_election.cc:304] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:47.725234 10483 raft_consensus.cc:2749] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:47.726473 10361 raft_consensus.cc:2468] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.728394 10176 leader_election.cc:304] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:47.729399 10525 raft_consensus.cc:2804] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.730077 10525 raft_consensus.cc:697] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:47.731169 10525 consensus_queue.cc:237] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.749836 10529 raft_consensus.cc:493] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.750649 10529 raft_consensus.cc:515] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.753582 10529 leader_election.cc:290] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.754468 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.755349 10287 raft_consensus.cc:2468] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.757455 10501 tablet_bootstrap.cc:492] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.757980 10501 ts_tablet_manager.cc:1403] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.060s	user 0.041s	sys 0.014s
I20260709 13:16:47.757797 10176 leader_election.cc:304] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:47.758751 10529 raft_consensus.cc:2804] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.760074 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20" is_pre_election: true
I20260709 13:16:47.762066 10529 raft_consensus.cc:493] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.762259 10361 raft_consensus.cc:2468] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 0.
I20260709 13:16:47.762606 10529 raft_consensus.cc:3060] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.759553 10111 catalog_manager.cc:5697] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.760846 10501 raft_consensus.cc:359] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.764715 10501 raft_consensus.cc:385] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.765072 10501 raft_consensus.cc:740] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.766041 10501 consensus_queue.cc:260] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.768107 10501 ts_tablet_manager.cc:1434] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:16:47.769174 10501 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: Bootstrap starting.
I20260709 13:16:47.772943 10529 raft_consensus.cc:515] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.774225 10501 tablet_bootstrap.cc:654] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:47.776455 10529 leader_election.cc:290] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:47.777642 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.778349 10287 raft_consensus.cc:3060] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.778721 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce140755ec84b65a672c00aff4e8f4c" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:47.779304 10361 raft_consensus.cc:3060] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.781713 10501 tablet_bootstrap.cc:492] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: No bootstrap required, opened a new log
I20260709 13:16:47.782315 10501 ts_tablet_manager.cc:1403] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:16:47.782207 10111 catalog_manager.cc:5697] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.785466 10501 raft_consensus.cc:359] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.786257 10501 raft_consensus.cc:385] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:47.786593 10361 raft_consensus.cc:2468] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.788240 10287 raft_consensus.cc:2468] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 1.
I20260709 13:16:47.789526 10176 leader_election.cc:304] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.786618 10501 raft_consensus.cc:740] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.790925 10529 raft_consensus.cc:2804] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.791570 10529 raft_consensus.cc:697] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:47.791697 10501 consensus_queue.cc:260] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.792459 10529 consensus_queue.cc:237] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.794129 10501 ts_tablet_manager.cc:1434] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:16:47.802789 10110 catalog_manager.cc:5697] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 1 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.864645 10408 raft_consensus.cc:493] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.865260 10408 raft_consensus.cc:515] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.867204 10408 leader_election.cc:290] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.867926 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.868301 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.869071 10287 raft_consensus.cc:2393] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c97b79fc3e2546ee92b3896deb664b20 in current term 1: Already voted for candidate 57da9927cb614902a03b2f62dca01939 in this term.
I20260709 13:16:47.870154 10325 leader_election.cc:304] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97b79fc3e2546ee92b3896deb664b20; no voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:47.870846 10521 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.877080 10521 raft_consensus.cc:2749] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:47.921083 10521 raft_consensus.cc:493] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.921613 10521 raft_consensus.cc:515] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.923813 10521 leader_election.cc:290] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.924598 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939" is_pre_election: true
I20260709 13:16:47.924930 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" is_pre_election: true
I20260709 13:16:47.925274 10212 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.925671 10287 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 0.
I20260709 13:16:47.926496 10325 leader_election.cc:304] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.928079 10521 raft_consensus.cc:2804] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:47.928499 10521 raft_consensus.cc:493] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.929016 10521 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.930619 10483 consensus_queue.cc:1048] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:47.937000 10521 raft_consensus.cc:515] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.939340 10521 leader_election.cc:290] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:47.940253 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:47.941033 10212 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.941289 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:47.942152 10287 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:47.949528 10212 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:47.951040 10325 leader_election.cc:304] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:47.951849 10287 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 1.
I20260709 13:16:47.952131 10521 raft_consensus.cc:2804] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:47.952699 10521 raft_consensus.cc:697] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:47.954191 10521 consensus_queue.cc:237] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:47.954909 10483 consensus_queue.cc:1048] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:47.965952 10111 catalog_manager.cc:5697] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 1 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.009516 10529 consensus_queue.cc:1048] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.020780 10478 consensus_queue.cc:1048] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.048673 10483 consensus_queue.cc:1048] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.056371 10287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
mode: GRACEFUL
new_leader_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:56692
I20260709 13:16:48.057169 10287 raft_consensus.cc:606] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Received request to transfer leadership to TS 57da9927cb614902a03b2f62dca01939
I20260709 13:16:48.059314 10524 consensus_queue.cc:1048] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:48.063673 10524 consensus_queue.cc:1048] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.076565 10287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
mode: GRACEFUL
new_leader_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:56692
I20260709 13:16:48.078848 10287 raft_consensus.cc:606] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Received request to transfer leadership to TS 57da9927cb614902a03b2f62dca01939
I20260709 13:16:48.087000 10483 consensus_queue.cc:1048] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.095476 10525 consensus_queue.cc:1048] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.100160 10524 raft_consensus.cc:993] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: : Instructing follower 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:48.100358 10361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:48.100754 10524 raft_consensus.cc:1081] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Signalling peer 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:48.101155 10361 raft_consensus.cc:606] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:48.103168 10211 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:60718
I20260709 13:16:48.103904 10211 raft_consensus.cc:493] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.104328 10211 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.108394 10212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed1b098347994e6094d96b8eef741247"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:16:48.109138 10212 raft_consensus.cc:606] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:48.113343 10211 raft_consensus.cc:515] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.114475 10528 consensus_queue.cc:1048] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.115792 10211 leader_election.cc:290] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:48.117566 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:48.118148 10360 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.118331 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e7e204958a642bba8e4842f16b99246" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:48.118933 10288 raft_consensus.cc:3055] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.119220 10288 raft_consensus.cc:740] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:48.119994 10288 consensus_queue.cc:260] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.121035 10362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
mode: GRACEFUL
new_leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:38284
I20260709 13:16:48.121259 10288 raft_consensus.cc:3060] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.121686 10362 raft_consensus.cc:606] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Received request to transfer leadership to TS 6c58a29b278c42f7a3dab0d07dd52a6d
I20260709 13:16:48.126060 10360 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:48.127307 10176 leader_election.cc:304] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:48.128494 10528 raft_consensus.cc:2804] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.129004 10528 raft_consensus.cc:697] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:48.129910 10528 consensus_queue.cc:237] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.131003 10288 raft_consensus.cc:2468] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:48.134948 10211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
mode: GRACEFUL
new_leader_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:16:48.135667 10211 raft_consensus.cc:606] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Received request to transfer leadership to TS c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:48.138546 10525 raft_consensus.cc:993] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: : Instructing follower c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:48.139063 10528 raft_consensus.cc:1081] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Signalling peer c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:48.140933 10361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:38276
I20260709 13:16:48.141387 10361 raft_consensus.cc:493] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.141705 10361 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.142462 10111 catalog_manager.cc:5697] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 1 to 2, leader changed from 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2) to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.147850 10361 raft_consensus.cc:515] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.149804 10361 leader_election.cc:290] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:48.152653 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:48.153560 10211 raft_consensus.cc:3055] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.153900 10211 raft_consensus.cc:740] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:48.154363 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0f0cd93c934b6eb96bdca6827fe1fe" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:48.155002 10288 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.154700 10211 consensus_queue.cc:260] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.156176 10211 raft_consensus.cc:3060] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.160415 10288 raft_consensus.cc:2468] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:48.161564 10211 raft_consensus.cc:2468] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:48.161564 10325 leader_election.cc:304] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c58a29b278c42f7a3dab0d07dd52a6d, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:48.162671 10521 raft_consensus.cc:2804] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.163187 10521 raft_consensus.cc:697] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:48.164112 10521 consensus_queue.cc:237] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.167090 10408 consensus_queue.cc:1048] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.173214 10111 catalog_manager.cc:5697] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 1 to 2, leader changed from 57da9927cb614902a03b2f62dca01939 (127.31.120.1) to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.178251 10408 raft_consensus.cc:993] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.178858 10408 raft_consensus.cc:1081] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.179828 10528 consensus_queue.cc:1048] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.188239 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56678
I20260709 13:16:48.188939 10288 raft_consensus.cc:493] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.189347 10288 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.194511 10528 consensus_queue.cc:1048] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:48.198808 10288 raft_consensus.cc:515] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.201784 10562 consensus_queue.cc:1048] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.201936 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:48.202576 10211 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.204937 10288 leader_election.cc:290] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:48.209074 10361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd35d27a47e64b6f9ab031abb6739943" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:48.209633 10361 raft_consensus.cc:3055] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.209939 10361 raft_consensus.cc:740] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:48.210798 10361 consensus_queue.cc:260] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.211869 10361 raft_consensus.cc:3060] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.214548 10211 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.215353 10212 raft_consensus.cc:1240] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Rejecting Update request from peer c97b79fc3e2546ee92b3896deb664b20 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:16:48.215394 10478 raft_consensus.cc:993] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.216624 10478 raft_consensus.cc:1081] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.218933 10251 leader_election.cc:304] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:48.219935 10286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed1b098347994e6094d96b8eef741247"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56662
I20260709 13:16:48.220693 10286 raft_consensus.cc:493] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.221127 10286 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.221280 10361 raft_consensus.cc:2468] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.222340 10524 raft_consensus.cc:2804] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.222884 10524 raft_consensus.cc:697] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:48.220889 10528 consensus_queue.cc:1048] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:48.223734 10524 consensus_queue.cc:237] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.229898 10286 raft_consensus.cc:515] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.231264 10287 raft_consensus.cc:1240] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Rejecting Update request from peer 57da9927cb614902a03b2f62dca01939 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:48.238561 10478 consensus_queue.cc:1048] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.240607 10528 consensus_queue.cc:1059] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.241374 10528 raft_consensus.cc:3055] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.241695 10528 raft_consensus.cc:740] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:48.242502 10528 consensus_queue.cc:260] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.243623 10528 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.244807 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:48.245756 10360 raft_consensus.cc:3060] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.247102 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1b098347994e6094d96b8eef741247" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:48.247728 10108 catalog_manager.cc:5697] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.260946 10286 leader_election.cc:290] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:48.260937 10211 raft_consensus.cc:2468] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.264907 10360 raft_consensus.cc:2468] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.265120 10251 leader_election.cc:304] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:48.266039 10524 raft_consensus.cc:2804] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.266695 10524 raft_consensus.cc:697] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:48.267498 10524 consensus_queue.cc:237] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.275104 10108 catalog_manager.cc:5697] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from 57da9927cb614902a03b2f62dca01939 (127.31.120.1) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.458168 10562 consensus_queue.cc:1048] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.470923 10563 consensus_queue.cc:1048] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.479038 10408 raft_consensus.cc:993] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: : Instructing follower 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.479543 10563 raft_consensus.cc:1081] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Signalling peer 6c58a29b278c42f7a3dab0d07dd52a6d to start an election
I20260709 13:16:48.481197 10286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
 from {username='slave'} at 127.0.0.1:56678
I20260709 13:16:48.481789 10286 raft_consensus.cc:493] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.482108 10286 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.487140 10286 raft_consensus.cc:515] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.489136 10286 leader_election.cc:290] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:48.489883 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:48.490286 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ff154cdf8c460a9b315cb0c88d3007" candidate_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:48.490921 10212 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.491039 10360 raft_consensus.cc:3055] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.491551 10360 raft_consensus.cc:740] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:48.492408 10360 consensus_queue.cc:260] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.493700 10360 raft_consensus.cc:3060] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.499150 10212 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.500444 10360 raft_consensus.cc:2468] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c58a29b278c42f7a3dab0d07dd52a6d in term 2.
I20260709 13:16:48.501071 10251 leader_election.cc:304] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, 6c58a29b278c42f7a3dab0d07dd52a6d; no voters: 
I20260709 13:16:48.501869 10524 raft_consensus.cc:2804] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.502404 10524 raft_consensus.cc:697] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Leader. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:48.503208 10524 consensus_queue.cc:237] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.509828 10108 catalog_manager.cc:5697] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d reported cstate change: term changed from 1 to 2, leader changed from c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3) to 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2). New cstate: current_term: 2 leader_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.559401 10286 raft_consensus.cc:1275] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.560670 10528 consensus_queue.cc:1048] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.568895 10360 raft_consensus.cc:1275] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.570524 10528 consensus_queue.cc:1048] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:48.610635 10500 raft_consensus.cc:993] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: : Instructing follower 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:48.611091 10524 raft_consensus.cc:1081] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Signalling peer 57da9927cb614902a03b2f62dca01939 to start an election
I20260709 13:16:48.612653 10212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
 from {username='slave'} at 127.0.0.1:60718
I20260709 13:16:48.613243 10212 raft_consensus.cc:493] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.613605 10212 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.618654 10212 raft_consensus.cc:515] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.620376 10212 leader_election.cc:290] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Requested vote from peers 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809), c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3:42141)
I20260709 13:16:48.621277 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:48.621585 10360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305c3498123f466ca2cee7964b6fcd9f" candidate_uuid: "57da9927cb614902a03b2f62dca01939" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
I20260709 13:16:48.622056 10286 raft_consensus.cc:3055] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.622437 10286 raft_consensus.cc:740] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:48.622408 10360 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.623567 10286 consensus_queue.cc:260] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.624802 10286 raft_consensus.cc:3060] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.630090 10360 raft_consensus.cc:2468] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:48.630964 10286 raft_consensus.cc:2468] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57da9927cb614902a03b2f62dca01939 in term 2.
I20260709 13:16:48.631480 10176 leader_election.cc:304] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:48.632421 10529 raft_consensus.cc:2804] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.632915 10529 raft_consensus.cc:697] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Becoming Leader. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:48.633783 10529 consensus_queue.cc:237] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:48.640501 10111 catalog_manager.cc:5697] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 reported cstate change: term changed from 1 to 2, leader changed from 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2) to 57da9927cb614902a03b2f62dca01939 (127.31.120.1). New cstate: current_term: 2 leader_uuid: "57da9927cb614902a03b2f62dca01939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.657508 10212 raft_consensus.cc:1275] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.658339 10211 raft_consensus.cc:1275] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.658921 10563 consensus_queue.cc:1048] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.659845 10524 consensus_queue.cc:1048] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.667507 10286 raft_consensus.cc:1275] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.671077 10360 raft_consensus.cc:1275] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.673036 10408 consensus_queue.cc:1048] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.673695 10483 consensus_queue.cc:1048] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.693666 10360 raft_consensus.cc:1275] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.695178 10571 consensus_queue.cc:1048] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.707067 10211 raft_consensus.cc:1275] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.708458 10524 consensus_queue.cc:1048] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:48.919215 10211 raft_consensus.cc:1275] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.920588 10524 consensus_queue.cc:1048] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.929572 10360 raft_consensus.cc:1275] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 6c58a29b278c42f7a3dab0d07dd52a6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.931674 10571 consensus_queue.cc:1048] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.152264 10360 raft_consensus.cc:1275] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.153618 10478 consensus_queue.cc:1048] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.162423 10286 raft_consensus.cc:1275] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer 57da9927cb614902a03b2f62dca01939: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.164233 10528 consensus_queue.cc:1048] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:51.138998 10111 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46884:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:16:51.159113 10111 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46884:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:16:51.183151 32224 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:51.315521 10592 log.cc:927] Time spent T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:16:51.391623 10620 tablet.cc:1724] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:16:51.436671 10251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.3:42141, user_credentials={real_user=slave}} blocked reactor thread for 46663.1us
W20260709 13:16:51.479676 10612 log.cc:927] Time spent T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:16:51.531649 10212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1"
dest_uuid: "57da9927cb614902a03b2f62dca01939"
mode: GRACEFUL
new_leader_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:16:51.532647 10212 raft_consensus.cc:606] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Received request to transfer leadership to TS c97b79fc3e2546ee92b3896deb664b20
W20260709 13:16:51.536810 10616 log.cc:927] Time spent T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
W20260709 13:16:51.541185 10614 log.cc:927] Time spent T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Append to log took a long time: real 0.170s	user 0.003s	sys 0.000s
W20260709 13:16:51.550345 10251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.31.120.1:37437, user_credentials={real_user=slave}} blocked reactor thread for 68937.4us
I20260709 13:16:51.531570 10632 tablet.cc:1724] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.493602 10624 tablet.cc:1724] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.588289 10643 tablet.cc:1724] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.592438 10635 raft_consensus.cc:993] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: : Instructing follower c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:51.592957 10635 raft_consensus.cc:1081] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Signalling peer c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:51.593639 10639 tablet.cc:1724] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.594856 10362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:38276
I20260709 13:16:51.595471 10362 raft_consensus.cc:493] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:51.595861 10362 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.532454 10634 tablet.cc:1724] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.605372 10647 tablet.cc:1724] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.606447 10620 tablet.cc:1724] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.608706 10646 tablet.cc:1724] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.620383 10648 tablet.cc:1724] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.622442 10639 tablet.cc:1724] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.618116 10362 raft_consensus.cc:515] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.630074 10362 leader_election.cc:290] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:51.591738 10645 tablet.cc:1724] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.633495 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:51.634124 10211 raft_consensus.cc:3055] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:51.634443 10211 raft_consensus.cc:740] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57da9927cb614902a03b2f62dca01939, State: Running, Role: LEADER
I20260709 13:16:51.635042 10624 tablet.cc:1724] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.635311 10211 consensus_queue.cc:260] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.617856 10642 tablet.cc:1724] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.643544 10649 tablet.cc:1724] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.645013 10211 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.624432 10643 tablet.cc:1724] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.669832 10286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f"
dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
mode: GRACEFUL
new_leader_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:49672
I20260709 13:16:51.666977 10634 tablet.cc:1724] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.682964 10645 tablet.cc:1724] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.671717 10286 raft_consensus.cc:606] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Received request to transfer leadership to TS c97b79fc3e2546ee92b3896deb664b20
I20260709 13:16:51.690196 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4b7ae8aa4c442eaf47c51d58dc7be1" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:51.690961 10284 raft_consensus.cc:3060] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.688966 10211 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:51.691957 10632 tablet.cc:1724] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.694765 32224 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:51.695518 32224 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:51.698124 10642 tablet.cc:1724] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.699621 10620 tablet.cc:1724] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.700249 10649 tablet.cc:1724] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.703483 10325 leader_election.cc:304] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:51.713382 10639 tablet.cc:1724] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.714474 10576 raft_consensus.cc:2804] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:51.705775 10648 tablet.cc:1724] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.714998 10576 raft_consensus.cc:697] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:51.716269 10576 consensus_queue.cc:237] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.724853 10647 tablet.cc:1724] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.728922 10284 raft_consensus.cc:2468] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 2.
I20260709 13:16:51.734948 10646 tablet.cc:1724] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.745407 10648 tablet.cc:1724] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:16:51.762130 10110 catalog_manager.cc:5697] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 1 to 2, leader changed from 57da9927cb614902a03b2f62dca01939 (127.31.120.1) to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 2 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:51.775789 10108 catalog_manager.cc:6242] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: soft_deleted_table [id=f5a4616581f24db3a1d6560a31f61391] alter complete (version 1)
I20260709 13:16:51.934887 10623 raft_consensus.cc:993] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: : Instructing follower c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:51.935395 10524 raft_consensus.cc:1081] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Signalling peer c97b79fc3e2546ee92b3896deb664b20 to start an election
I20260709 13:16:51.936944 10362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f"
dest_uuid: "c97b79fc3e2546ee92b3896deb664b20"
 from {username='slave'} at 127.0.0.1:38280
I20260709 13:16:51.937433 10362 raft_consensus.cc:493] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:51.937736 10362 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.943569 10362 raft_consensus.cc:515] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.945498 10362 leader_election.cc:290] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 3 election: Requested vote from peers 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437), 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809)
I20260709 13:16:51.946254 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "57da9927cb614902a03b2f62dca01939"
I20260709 13:16:51.946549 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ff0a1b9b334e3287ba2887b852ba4f" candidate_uuid: "c97b79fc3e2546ee92b3896deb664b20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d"
I20260709 13:16:51.947021 10211 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.947214 10284 raft_consensus.cc:3055] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:51.947664 10284 raft_consensus.cc:740] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c58a29b278c42f7a3dab0d07dd52a6d, State: Running, Role: LEADER
I20260709 13:16:51.948438 10284 consensus_queue.cc:260] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.949447 10284 raft_consensus.cc:3060] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.952297 10211 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 3.
I20260709 13:16:51.953418 10325 leader_election.cc:304] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57da9927cb614902a03b2f62dca01939, c97b79fc3e2546ee92b3896deb664b20; no voters: 
I20260709 13:16:51.954155 10576 raft_consensus.cc:2804] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:51.954663 10576 raft_consensus.cc:697] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 3 LEADER]: Becoming Leader. State: Replica: c97b79fc3e2546ee92b3896deb664b20, State: Running, Role: LEADER
I20260709 13:16:51.955494 10284 raft_consensus.cc:2468] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97b79fc3e2546ee92b3896deb664b20 in term 3.
I20260709 13:16:51.955423 10576 consensus_queue.cc:237] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } }
I20260709 13:16:51.964586 10110 catalog_manager.cc:5697] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 reported cstate change: term changed from 2 to 3, leader changed from 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2) to c97b79fc3e2546ee92b3896deb664b20 (127.31.120.3). New cstate: current_term: 3 leader_uuid: "c97b79fc3e2546ee92b3896deb664b20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c97b79fc3e2546ee92b3896deb664b20" member_type: VOTER last_known_addr { host: "127.31.120.3" port: 42141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.096009 10211 raft_consensus.cc:1275] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.097273 10576 consensus_queue.cc:1048] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.107774 10284 raft_consensus.cc:1275] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.112731 10576 consensus_queue.cc:1048] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.387212 10284 raft_consensus.cc:1275] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 3 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:52.389050 10660 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c58a29b278c42f7a3dab0d07dd52a6d" member_type: VOTER last_known_addr { host: "127.31.120.2" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.398222 10211 raft_consensus.cc:1275] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 3 FOLLOWER]: Refusing update from remote peer c97b79fc3e2546ee92b3896deb664b20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:52.403302 10605 consensus_queue.cc:1048] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57da9927cb614902a03b2f62dca01939" member_type: VOTER last_known_addr { host: "127.31.120.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.755859 32224 tablet_server.cc:179] TabletServer@127.31.120.1:0 shutting down...
I20260709 13:16:52.782848 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:52.783574 32224 tablet_replica.cc:333] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.784289 32224 raft_consensus.cc:2243] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.784787 32224 raft_consensus.cc:2272] T fd35d27a47e64b6f9ab031abb6739943 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.786720 32224 tablet_replica.cc:333] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.787297 32224 raft_consensus.cc:2243] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.787796 32224 raft_consensus.cc:2272] T f9240a430c924d8788abe52364490dc3 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.789529 32224 tablet_replica.cc:333] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.790055 32224 raft_consensus.cc:2243] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.790482 32224 raft_consensus.cc:2272] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.792276 32224 tablet_replica.cc:333] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.792807 32224 raft_consensus.cc:2243] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.793572 32224 raft_consensus.cc:2272] T f0ffef919b39443b8acad51477e127d2 P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.795953 32224 tablet_replica.cc:333] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.796859 32224 raft_consensus.cc:2243] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.797391 32224 raft_consensus.cc:2272] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.800030 32224 tablet_replica.cc:333] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.800622 32224 raft_consensus.cc:2243] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.801035 32224 raft_consensus.cc:2272] T df5108da9a84462c80d89f0fad32ca6f P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.802906 32224 tablet_replica.cc:333] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.803439 32224 raft_consensus.cc:2243] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.803861 32224 raft_consensus.cc:2272] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.805821 32224 tablet_replica.cc:333] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.806326 32224 raft_consensus.cc:2243] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.807073 32224 raft_consensus.cc:2272] T 59a2f411ec384b428052c5dbf957e464 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.809110 32224 tablet_replica.cc:333] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.809676 32224 raft_consensus.cc:2243] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.810096 32224 raft_consensus.cc:2272] T ed1b098347994e6094d96b8eef741247 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.812280 32224 tablet_replica.cc:333] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.812793 32224 raft_consensus.cc:2243] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.813531 32224 raft_consensus.cc:2272] T 305c3498123f466ca2cee7964b6fcd9f P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.815344 32224 tablet_replica.cc:333] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.815927 32224 raft_consensus.cc:2243] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.816354 32224 raft_consensus.cc:2272] T e3ff0a1b9b334e3287ba2887b852ba4f P 57da9927cb614902a03b2f62dca01939 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.819368 32224 tablet_replica.cc:333] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.819967 32224 raft_consensus.cc:2243] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.820463 32224 raft_consensus.cc:2272] T 17ff154cdf8c460a9b315cb0c88d3007 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.822299 32224 tablet_replica.cc:333] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.822814 32224 raft_consensus.cc:2243] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.823587 32224 raft_consensus.cc:2272] T 0e7e204958a642bba8e4842f16b99246 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.825477 32224 tablet_replica.cc:333] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.826001 32224 raft_consensus.cc:2243] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.826459 32224 raft_consensus.cc:2272] T e8bd4b8feb90401d9ad536672e687d90 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.828522 32224 tablet_replica.cc:333] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.829034 32224 raft_consensus.cc:2243] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.829804 32224 raft_consensus.cc:2272] T 1ce140755ec84b65a672c00aff4e8f4c P 57da9927cb614902a03b2f62dca01939 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.831046 10325 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
I20260709 13:16:52.831898 32224 tablet_replica.cc:333] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.832602 32224 raft_consensus.cc:2243] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.833010 32224 raft_consensus.cc:2272] T e92a28ff744a4d46b44d77f338a207fe P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.834201 10325 consensus_peers.cc:597] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 -> Peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Couldn't send request to peer 57da9927cb614902a03b2f62dca01939. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.835260 32224 tablet_replica.cc:333] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.835922 32224 raft_consensus.cc:2243] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.836691 32224 raft_consensus.cc:2272] T 6fee7a135f4a4253a72e695a15cf216a P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.838646 32224 tablet_replica.cc:333] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939: stopping tablet replica
I20260709 13:16:52.839111 32224 raft_consensus.cc:2243] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.839936 32224 raft_consensus.cc:2272] T 6a86931f91a949088cffe4d239de9f49 P 57da9927cb614902a03b2f62dca01939 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.846925 10325 consensus_peers.cc:597] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 -> Peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Couldn't send request to peer 57da9927cb614902a03b2f62dca01939. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.870476 32224 tablet_server.cc:196] TabletServer@127.31.120.1:0 shutdown complete.
I20260709 13:16:52.905318 32224 tablet_server.cc:179] TabletServer@127.31.120.2:0 shutting down...
W20260709 13:16:52.919518 10325 consensus_peers.cc:597] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 -> Peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Couldn't send request to peer 57da9927cb614902a03b2f62dca01939. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.920236 10251 consensus_peers.cc:597] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d -> Peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Couldn't send request to peer 57da9927cb614902a03b2f62dca01939. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.938351 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:52.939296 32224 tablet_replica.cc:333] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.940295 32224 raft_consensus.cc:2243] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.941179 32224 raft_consensus.cc:2272] T fd35d27a47e64b6f9ab031abb6739943 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.943058 32224 tablet_replica.cc:333] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.943728 32224 raft_consensus.cc:2243] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.944461 32224 raft_consensus.cc:2272] T f9240a430c924d8788abe52364490dc3 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.946235 32224 tablet_replica.cc:333] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.946745 32224 raft_consensus.cc:2243] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.947127 32224 raft_consensus.cc:2272] T 6a86931f91a949088cffe4d239de9f49 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.949101 32224 tablet_replica.cc:333] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.949613 32224 raft_consensus.cc:2243] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.949965 32224 raft_consensus.cc:2272] T 6fee7a135f4a4253a72e695a15cf216a P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.952028 32224 tablet_replica.cc:333] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.952492 32224 raft_consensus.cc:2243] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.952859 32224 raft_consensus.cc:2272] T 5d0f0cd93c934b6eb96bdca6827fe1fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.954516 32224 tablet_replica.cc:333] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.955003 32224 raft_consensus.cc:2243] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.955371 32224 raft_consensus.cc:2272] T f0ffef919b39443b8acad51477e127d2 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.957070 32224 tablet_replica.cc:333] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.957532 32224 raft_consensus.cc:2243] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.958237 32224 raft_consensus.cc:2272] T df5108da9a84462c80d89f0fad32ca6f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.960180 32224 tablet_replica.cc:333] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.960665 32224 raft_consensus.cc:2243] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.961020 32224 raft_consensus.cc:2272] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.963373 32224 tablet_replica.cc:333] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.964229 32224 raft_consensus.cc:2243] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.965094 32224 raft_consensus.cc:2272] T 2d68b6b0712c4812a4c3dd5a8c09c46c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.967003 32224 tablet_replica.cc:333] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.967585 32224 raft_consensus.cc:2243] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.968046 32224 raft_consensus.cc:2272] T 59a2f411ec384b428052c5dbf957e464 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.969897 32224 tablet_replica.cc:333] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.970366 32224 raft_consensus.cc:2243] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.971051 32224 raft_consensus.cc:2272] T ed1b098347994e6094d96b8eef741247 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.972905 32224 tablet_replica.cc:333] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.973471 32224 raft_consensus.cc:2243] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.973901 32224 raft_consensus.cc:2272] T 305c3498123f466ca2cee7964b6fcd9f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.975818 32224 tablet_replica.cc:333] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.976540 32224 raft_consensus.cc:2243] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.977039 32224 raft_consensus.cc:2272] T e3ff0a1b9b334e3287ba2887b852ba4f P 6c58a29b278c42f7a3dab0d07dd52a6d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.978871 10325 consensus_peers.cc:597] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 -> Peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Couldn't send request to peer 6c58a29b278c42f7a3dab0d07dd52a6d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.980350 32224 tablet_replica.cc:333] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.981043 32224 raft_consensus.cc:2243] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.981854 32224 raft_consensus.cc:2272] T 17ff154cdf8c460a9b315cb0c88d3007 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.983898 32224 tablet_replica.cc:333] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.984757 32224 raft_consensus.cc:2243] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.985893 32224 raft_consensus.cc:2272] T e8bd4b8feb90401d9ad536672e687d90 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.988708 32224 tablet_replica.cc:333] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.989266 32224 raft_consensus.cc:2243] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.989806 32224 raft_consensus.cc:2272] T 0e7e204958a642bba8e4842f16b99246 P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.992121 32224 tablet_replica.cc:333] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.992902 32224 raft_consensus.cc:2243] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.993364 32224 raft_consensus.cc:2272] T 1ce140755ec84b65a672c00aff4e8f4c P 6c58a29b278c42f7a3dab0d07dd52a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.995167 32224 tablet_replica.cc:333] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d: stopping tablet replica
I20260709 13:16:52.995741 32224 raft_consensus.cc:2243] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.996172 32224 raft_consensus.cc:2272] T e92a28ff744a4d46b44d77f338a207fe P 6c58a29b278c42f7a3dab0d07dd52a6d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:53.023239 10325 consensus_peers.cc:597] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 -> Peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Couldn't send request to peer 6c58a29b278c42f7a3dab0d07dd52a6d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:40809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:53.026443 32224 tablet_server.cc:196] TabletServer@127.31.120.2:0 shutdown complete.
I20260709 13:16:53.058081 32224 tablet_server.cc:179] TabletServer@127.31.120.3:0 shutting down...
I20260709 13:16:53.083560 32224 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:53.084388 32224 tablet_replica.cc:333] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.085367 32224 raft_consensus.cc:2243] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.086054 32224 raft_consensus.cc:2272] T f9240a430c924d8788abe52364490dc3 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.088135 32224 tablet_replica.cc:333] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
W20260709 13:16:53.088585 10325 consensus_peers.cc:597] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 -> Peer 57da9927cb614902a03b2f62dca01939 (127.31.120.1:37437): Couldn't send request to peer 57da9927cb614902a03b2f62dca01939. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.1:37437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:53.088932 32224 raft_consensus.cc:2243] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.089521 32224 raft_consensus.cc:2272] T fd35d27a47e64b6f9ab031abb6739943 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.091401 32224 tablet_replica.cc:333] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.092006 32224 raft_consensus.cc:2243] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.092422 32224 raft_consensus.cc:2272] T ed1b098347994e6094d96b8eef741247 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.094183 32224 tablet_replica.cc:333] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.094784 32224 raft_consensus.cc:2243] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:53.095595 32224 raft_consensus.cc:2272] T e3ff0a1b9b334e3287ba2887b852ba4f P c97b79fc3e2546ee92b3896deb664b20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.098016 32224 tablet_replica.cc:333] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.098599 32224 raft_consensus.cc:2243] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.099045 32224 raft_consensus.cc:2272] T 305c3498123f466ca2cee7964b6fcd9f P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.100885 32224 tablet_replica.cc:333] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.101431 32224 raft_consensus.cc:2243] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:53.102195 32224 raft_consensus.cc:2272] T 5d0f0cd93c934b6eb96bdca6827fe1fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.104322 32224 tablet_replica.cc:333] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.105195 32224 raft_consensus.cc:2243] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:53.106217 32224 raft_consensus.cc:2272] T 2b4b7ae8aa4c442eaf47c51d58dc7be1 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.108886 32224 tablet_replica.cc:333] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.109493 32224 raft_consensus.cc:2243] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.109984 32224 raft_consensus.cc:2272] T f0ffef919b39443b8acad51477e127d2 P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:53.110867 10325 consensus_peers.cc:597] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 -> Peer 6c58a29b278c42f7a3dab0d07dd52a6d (127.31.120.2:40809): Couldn't send request to peer 6c58a29b278c42f7a3dab0d07dd52a6d. Status: Network error: Client connection negotiation failed: client connection to 127.31.120.2:40809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:53.112094 32224 tablet_replica.cc:333] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.112758 32224 raft_consensus.cc:2243] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.113198 32224 raft_consensus.cc:2272] T df5108da9a84462c80d89f0fad32ca6f P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.115284 32224 tablet_replica.cc:333] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.115918 32224 raft_consensus.cc:2243] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.116403 32224 raft_consensus.cc:2272] T 6fee7a135f4a4253a72e695a15cf216a P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.118633 32224 tablet_replica.cc:333] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.119161 32224 raft_consensus.cc:2243] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.119645 32224 raft_consensus.cc:2272] T 6a86931f91a949088cffe4d239de9f49 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.121727 32224 tablet_replica.cc:333] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.122259 32224 raft_consensus.cc:2243] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.122697 32224 raft_consensus.cc:2272] T 17ff154cdf8c460a9b315cb0c88d3007 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.124588 32224 tablet_replica.cc:333] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.125137 32224 raft_consensus.cc:2243] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.125603 32224 raft_consensus.cc:2272] T 0e7e204958a642bba8e4842f16b99246 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.127449 32224 tablet_replica.cc:333] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.128019 32224 raft_consensus.cc:2243] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.128506 32224 raft_consensus.cc:2272] T e8bd4b8feb90401d9ad536672e687d90 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.130481 32224 tablet_replica.cc:333] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.130997 32224 raft_consensus.cc:2243] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.131453 32224 raft_consensus.cc:2272] T 1ce140755ec84b65a672c00aff4e8f4c P c97b79fc3e2546ee92b3896deb664b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.133250 32224 tablet_replica.cc:333] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.133766 32224 raft_consensus.cc:2243] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:53.134614 32224 raft_consensus.cc:2272] T e92a28ff744a4d46b44d77f338a207fe P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.136798 32224 tablet_replica.cc:333] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.137323 32224 raft_consensus.cc:2243] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.137784 32224 raft_consensus.cc:2272] T 2d68b6b0712c4812a4c3dd5a8c09c46c P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.139850 32224 tablet_replica.cc:333] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20: stopping tablet replica
I20260709 13:16:53.140336 32224 raft_consensus.cc:2243] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.140780 32224 raft_consensus.cc:2272] T 59a2f411ec384b428052c5dbf957e464 P c97b79fc3e2546ee92b3896deb664b20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.171407 32224 tablet_server.cc:196] TabletServer@127.31.120.3:0 shutdown complete.
I20260709 13:16:53.201997 32224 master.cc:562] Master@127.31.120.62:42313 shutting down...
I20260709 13:16:53.223986 32224 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:53.224576 32224 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.224918 32224 tablet_replica.cc:333] T 00000000000000000000000000000000 P b5eaf6252ad5425292938d766398107d: stopping tablet replica
I20260709 13:16:53.246840 32224 master.cc:584] Master@127.31.120.62:42313 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11509 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24013 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (298072 ms total)
[  PASSED  ] 8 tests.
I20260709 13:16:53.301792 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 393 messages since previous log ~14 seconds ago
I20260709 13:16:53.302065 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 393 messages since previous log ~14 seconds ago
I20260709 13:16:53.302282 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 392 messages since previous log ~14 seconds ago
I20260709 13:16:53.302505 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:16:53.302704 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:16:53.302938 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 4 messages since previous log ~125 seconds ago
I20260709 13:16:53.303136 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~145 seconds ago
I20260709 13:16:53.303401 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~0 seconds ago
I20260709 13:16:53.303972 32224 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
